Keystroke Macro Recorder

Ask about the individual Gambas components here.
User avatar
BruceSteers
Posts: 521
Joined: Thursday 23rd July 2020 5:20pm

Keystroke Macro Recorder

Post by BruceSteers » Thursday 05th November 2020 1:21pm

Has anyone tried to make one?

I have but it's not been easy. got a few limitations on the Alt/AltGr/Shift-AltGr keys for non standard ascii chars I've been trying to get my head around.

Using the keysym file doesn't seem to work.

I tried this code...
Select Key.Code
Case key["Alt_L"]
 Return "[Alt_L]"
Case Key["ISO_Level3_Shift"]
 Return "[ISO_Level3_Shift]"  ' AltGr
Case Key["ISO_Level5_Shift"]
 Return "[ISO_Level5_Shift]"  ' AltGr+Shif
End Select
But it does not seem to work,
i have been examining the String.Code() values instead to see if values >127 for
ISO_Level_3_Shift (AltGr range) but i believe..
ISO_Level_5_Shift is Shift-AltGr but i don't know where the value crosses over. And i'm not even sure i'm understanding everything correctly.
ISO_Level_3_Shift seems to work for Things like Benoíts í char that is just AltGr + i to for the most part it's working but i think here is where it will error with Shift-AltGr.

But suffice to say a MacroRecorder.class that records and plays back keystrokes has been made in this project...


This is my scripting text editor beta
Features..
Uses TextEditor and utilises "sh" "gambas" highlight styles/modes
highlighting is fully customisable and saves a config for each mode.
Has a run button to run script in TerminalView , works on bash scripts and gambas scripts
Multiple Tabs (Experimental)

Latest Addition..
Keystroke Macro Recorder.
Very much beta and needs lots of testing.
The app has temporary buttons at present to start/stop/play/clear the macro.
It will recognise an F3 (Find Next) keypress that is handy in macro recording

MacroRecorder.class should be addable to most projects, it's not TextEditor only
Its set up like this...
NOTE: Obsolete method for new version
Public Sub MenuMacroStart_Click()

If Not mMacro Then mMacro = New MacroRecorder

mMacro.Editor = $tEditor

mMacro.Record()

End
$tEditor Object can theoretically be anything but ive only tested it on a texteditor so far. it sets up a Key reading event hooks and needs to set focus on the object before sending keys so object must be set. should work on anything that gives and recieves key down and keyup events.

Todo...
Load some default macros.
Test and bugfix the s**t out of it.
Try to get Benoít and the team to refine it and add it to gambas because i would soooo love a macro recorder in gambas :)

Please feel free to suggest anything you might know i've missed or have not understood about keys.
Or any improvements you might offer.
Thank you for listening..

All the best
Bruce
Last edited by BruceSteers on Saturday 14th November 2020 4:27am, edited 4 times in total.
Wishing well
Bruce

If at first you don't succeed , try it differently.

User avatar
BruceSteers
Posts: 521
Joined: Thursday 23rd July 2020 5:20pm

Re: Keystroke Macro Recorder

Post by BruceSteers » Thursday 05th November 2020 3:11pm

Also todo
add a wayland detection/handling , possibly sendkey alternative because att present it only works for x11 not wayland.
Wishing well
Bruce

If at first you don't succeed , try it differently.

User avatar
cogier
Site Admin
Posts: 721
Joined: Wednesday 21st September 2016 2:22pm
Location: Guernsey, Channel Islands

Re: Keystroke Macro Recorder

Post by cogier » Thursday 05th November 2020 4:36pm

Wow! You have been busy. I don't have the answers you requested but I have some suggestions, please feel free to ignore them.

1/. Try adding some 'Padding' (5 to 10) to the FMain Form.
2/. Should the FMain.text = "+"?
3/. In the edit menu is 'Prefrences' when I think you mean 'Preferences'.
4/. I need a 'Help' section as this program is quite complex.
5/. The 'Balloon' is not very readable and 'result' is spelt incorrectly.

Image

User avatar
BruceSteers
Posts: 521
Joined: Thursday 23rd July 2020 5:20pm

Re: Keystroke Macro Recorder

Post by BruceSteers » Thursday 05th November 2020 6:05pm

cogier wrote:
Thursday 05th November 2020 4:36pm
Wow! You have been busy. I don't have the answers you requested but I have some suggestions, please feel free to ignore them.

1/. Try adding some 'Padding' (5 to 10) to the FMain Form.
2/. Should the FMain.text = "+"?
3/. In the edit menu is 'Prefrences' when I think you mean 'Preferences'.
4/. I need a 'Help' section as this program is quite complex.
5/. The 'Balloon' is not very readable and 'result' is spelt incorrectly.

Image
I did the Padding cheers.
Changed Title as i've been testing it i have been running it with file path args.
And , haha , typo's lol i got big chunky man hands, i should be a builder not a programmer ;)

As for the app and help it's far from release ready so hadn't got that far yet.
It's not too complex :), here;s some need-to-know stuff...

It's main function is to be a Scripting tool so...
it auto-detects filetypes bash scripts and gambas scripts and sets TextEditor highligh and mode accordingly.
Highlight colours and bold/underline can be set independently. Ie, a gambas colour scheme will save and be different to your sh colour scheme. see my post about adding the highlight to get a description on styles.

It has a run button (or press f5) , this runs the script in a TerminalView
Buttons are, Open, Reload Save, Save as, Checkbox +x chmod, Run Script and New Tab.

On a bash scriipt and gambas script it will detect functions and populate the list on the right
(See attached image of how it looks on my Gambas_Comiple script that has loads of functions and is a ball ache scrolling up and down the file to find them ,, hence the handy list. :) )

The start/stop/play/clear buttons are temporary put in there to quickly launch the macro recorder but i may nice it up a bit and make it show/hide in settings. (there's also key shortcuts for macro functions)

Notes on if you want to use the MacroRecorder.class in your own app.
The F3 (find next) function of the editor is priceless in a macro recorder but the recorder
will not catch the key press event as the main app catches it first.

So i've added a function to manually add codes.
I put this code in the Btn_Search_Click() event that the menu function calls when F3 is pressed
If mMacro Then
  If mMacro.State = mMacro.Recording Then mMacro.AddKeySym("F3")
Endif
KeySym string must be from the /usr/include/keysymdef.h file not gambas ones (remove XK_ prefix)
You would need to do the same for any other keycode that you'd have in your app but want the macro recorder to pick up

This editor is not really ready for release but i wanted to show people the macro recorder in action.
see if anyone can break it and moan about it , or bug report as we like to say ;)
Or expert advice , it's all good in the long run :)
My main intention here with uploading the program was to get some guidance on my recording macro methods.

Cheers for spotting those bugs though , i do love to squish bugs ;)

macro class file has got lots of 'Debug blah; blah lines.
So search and replace 'Debug with Debug and you'll see all the macro workings.

Cheers for looking Charlie , if you have a personal editor you've made or anything you think might be good to have a macro recorder please add it.

It's on gitLab , feel free to fork and offer commits.
I'm going to delete and remake the repository soon though as i typo'd the name when i made it and i've changed the name to scripted but the repo path is still sripted lol. :lol:

And as for the balloons , balloons just use tooltip colours , that's your settings m8, i've not done anything towards balloon style ;)
I know how to add colour setting to gambas balloons though ;) i've done it before :)
All the best
Bruce
Attachments
snap.png
snap.png (312.47 KiB) Viewed 1977 times
Wishing well
Bruce

If at first you don't succeed , try it differently.

User avatar
BruceSteers
Posts: 521
Joined: Thursday 23rd July 2020 5:20pm

Re: Keystroke Macro Recorder

Post by BruceSteers » Sunday 08th November 2020 4:30am

Changed how the MacroRecorder.class works. quite a bit.

For starters i did away with using any gambas Key class naming stuff.
The x11 SendKey function works better and i get much more scope if i use the definitions from the keysymdef.h file.
So now it reads in the whole definitions file in and gets all the key codes not just the few added to gambas.
And made the MacroRecorder.class much smaller as i was able to remove all the code trying to convert

I initially went the route of the class setting up it's own KeyPress and KeyRelease event handlers but it didn't work to my liking.
setting up an Object.Attach to the text object stopped all other events triggering in the main application while the recorder was running (like Change() and Activate() events.

so now it works a bit different..
You no longer provide it a text object at all except for the MacroRecorder.Play and the only reason for that is so the macro calls it's .SetFocus() method before running.

I realised there was no real need to use the text object.
The Key class data is virtual and readable across classes so I though it better to do it a different way.

Instead now to record the key events you use MacroRecorder.Press() and MacroRecorder.Release() functions in your apps own key handlers.

So to initialize do something like the following..
(PS. I also added a static function CanWork() that tests you are not using wayland and that the keysymdef include file is present.)

Public mMacro as MacroRecorder

Public Sub Form_Load()

If MacroRecorder.CanWork() Then 
  mMacro = New MacroRecorder
  Me.HBoxMacro.Enabled = True  '  enable the macro buttons
Else 
  Me.HBoxMacro.Enabled = False
Endif


End

Public Sub TextEditor1_KeyRelease()

  If mMacro.State = mMacro.Recording Then mMacro.Release()
  
' your code here..

End

Public Sub TextEditor1_KeyPress()

If mMacro.State = mMacro.Recording Then mMacro.Press()

' your code here..

End

Manage the recordings using Start() Stop() Play() and Clear()

Play() has one needed argument and one optional one.
Public Sub Play($TextObject As Object, Optional PrintOnly As Boolean) As Boolean
Textobject is the object you want the macro to play in.
if the optional PrintOnly is set to True the saved list of keys is just printed to stdout and presses are not produced.

Recordings can be accessed with the mMacro.Buffer and mMacro.Modkey arrays.
Each Key press or release recorded has the following data (n being the array pos).
Buffer[n] ' the data, either a single Char or a keysymdef code
ModKey[n][IsModKey,DownPress] ' a boolean[] array 1st element is if it's a modifier key or not 2nd true for kedown false for keyup.

Only modifier keys record both key down and key up events, any other key will be a key down and instantly up again event but as modifiers can be held down they record both events as and when they occur.

the class can be downloaded independently from the testing branch of the Editor on gitlab .
https://gitlab.com/bsteers4/sripted/-/tree/testing

Sorry still not written any docs.
Enjoy.
Bruce
Last edited by BruceSteers on Monday 09th November 2020 11:57pm, edited 2 times in total.
Wishing well
Bruce

If at first you don't succeed , try it differently.

User avatar
cogier
Site Admin
Posts: 721
Joined: Wednesday 21st September 2016 2:22pm
Location: Guernsey, Channel Islands

Re: Keystroke Macro Recorder

Post by cogier » Sunday 08th November 2020 3:47pm

I got into trouble straight away. I thought I would record a macro (Tools > Macro recorder(Empty) > Start recording). Result below: -

Image

EDIT
Pressing the "Start" button brings up the same error.

EDIT
I noticed this: -
Image

If you add 'FuncGview.Columns.Width = -1' line after line 50 you get this: -
Image

User avatar
BruceSteers
Posts: 521
Joined: Thursday 23rd July 2020 5:20pm

Re: Keystroke Macro Recorder

Post by BruceSteers » Sunday 08th November 2020 7:04pm

Sorry fella i thought i explained it.

I have done away with MacroRecorder.Editor property. it no longer exists as is not really needed.

Only time you supply the text editor or textarea object now is when using MacroRecorder.Play() method.

The new version of the editor uploaded with the new macro recorder should work.
did you just pop the macro class file in the old verison source?

that wont work, had a big re-write in how it works.

Attached new version...
Last edited by BruceSteers on Saturday 14th November 2020 4:27am, edited 1 time in total.
Wishing well
Bruce

If at first you don't succeed , try it differently.

User avatar
BruceSteers
Posts: 521
Joined: Thursday 23rd July 2020 5:20pm

Re: Keystroke Macro Recorder

Post by BruceSteers » Sunday 08th November 2020 8:18pm

If you add 'FuncGview.Columns.Width = -1' line after line 50 you get this: -
Cool, cheers Charlie , i had no idea i could do that :)
Wishing well
Bruce

If at first you don't succeed , try it differently.

User avatar
BruceSteers
Posts: 521
Joined: Thursday 23rd July 2020 5:20pm

Re: Keystroke Macro Recorder

Post by BruceSteers » Tuesday 10th November 2020 2:40am

Right , new discovery.

Seems using buttons to play the macro is a bad idea.
If you click the button too rapidly it sometimes prints wrong.
I think it has to do with the editors focus.
if it does not have focus it does not receive the macro keys. and if you're clicking off the page while it's playing it messes up :(
I put loads of calls to .SetFocus() in the play function but it still can miss some stuff if you are clicking away like a nutter on the button.

Sooo.
Away with the buttons and will just go for the menu/shortcuts as it's okay when launching from the menu and also when pressing the play menu hotkey (F7)

I recorded a little video to show it in action.
In the video it's making short work of something i actually did for the first draft of this class.
Well I got part way though and loaded up JEdit as it's the only editor i have with macro recorder.
Oh the irony of wishing i had a macro recorder to help me make a macro recorder :) lol

I record a macro, then play it from the menu, then hit F7 a few times......
:)
http://bws.org.uk/simplescreenrecorder- ... .22.03.mp4
Wishing well
Bruce

If at first you don't succeed , try it differently.

User avatar
BruceSteers
Posts: 521
Joined: Thursday 23rd July 2020 5:20pm

Re: Keystroke Macro Recorder

Post by BruceSteers » Thursday 12th November 2020 3:09pm

Last night i was a fool :(

I have no idea what came over me but in the process of deleting my scripted git repository because i initially misspelt it a sripted :-\
Somehow while doing that and refreshing my local dirs i managed to delete all changes since the last beta i posted, (about 4 days work :'( )

I almost cried.

So i've taken the link down and am rebuilding it all now as theres some big issues.
Mostly i just discovered the keycodes do not get recognised in QT.
I am using the reccommended wiki method of using the keysymsdef.h files definitions against the key.code and on gtk it works fine but the codes do not match anything in qt.
So once i'm happy with how it's working i'll re-upload and for now the ScriptED app will only use gtk2

Thought it worth mentioning if you have thought of using the class.
currently does not support QT
It could be a bug in gambas
the wiki says don't use code directly use keysymsdef method.
will report it in the m/l.
Wishing well
Bruce

If at first you don't succeed , try it differently.

Post Reply