Object serialization, allowing save and load of class structure data

Post Reply
User avatar
Godzilla
Posts: 35
Joined: Wednesday 26th September 2018 11:20am

Object serialization, allowing save and load of class structure data

Post by Godzilla » Monday 07th October 2019 8:38am

This is a pending solution to the problem I had with my experiment with the Task method (Task_Test) in my other thread, where data contained inside structured class arrays would mysteriously vanish after the Tasks/Forks had completed their cycles.

However, I felt this object serialization code, in and of itself, is so nifty and cool, I felt it deserved its own thread. People who may be looking for something like this probably aren't going to find it buried in a seemingly unrelated thread.

This code was posted by Jussi Lahtinen in 2013. I've put it in a project and modified it to handle Date variables, and to identify any unknown variable types it may encounter, using the Print command. It isn't complete in the sense that it handles all variables, but can easily be modified to do so.

The reason I call this a "pending" solution to the Task_Test problem is because the LoadValues and SaveValues object serialization functions need to be able to handle not just a structured class, but a structured class array. I'm not quite sure how to approach that. So I thought I'd post a project example here, which saves and loads a non-array class structure. With a request for someone to check out the project and modify it to work with arrays as well.

Then, I'll use it to modify the Task_Test project in my other thread to fix the bug and get it fully functional.

Here's some background as to why Task_Test fails, and I'll also include this info in the other thread along with a working project.

To quote from a Perl thread regarding this exact same problem someone encountered with using Fork:
It's not a matter of scoping, it's a matter of different processes. Parallel::ForkManager uses fork() (hence the name). This means that each version running in parallel is actually a separate process (a separate invocation of the perl interpreter) and thus separate memory. The variables will have the same name in each process, but they won't point to the same place in memory.
So in reality, the variables inside Task_Test aren't even being altered by the Task method. Task copies these processes and hands them over to the system (allowing the bypassing of the 1 thread limitation of Gambas). The system completes these tasks and then they vanish, along with all their variables and any alterations done to them. The Gambas program, and the variables within it, remain unchanged for all practical purposes. That's why variables that were seemingly packed with data a microsecond before apparently return back to their empty undeclared state immediately. They've not actually been touched.

One solution to this problem is to use the same solution cogier came up with, allowing a parent program to communicate to and from satellite programs, which was to use File.Save and File.Load. The problem with this in the case of Task_Test is: the structured class arrays I use are too complex to be handled by File.Save and File.Load. Object serialization is the solution, and a nifty one at that.

Thanks for any help. Here's the example project.
Attachments
Object_Serialization.tar.gz
(14.93 KiB) Downloaded 7 times

User avatar
Godzilla
Posts: 35
Joined: Wednesday 26th September 2018 11:20am

Re: Object serialization, allowing save and load of class structure data

Post by Godzilla » Tuesday 08th October 2019 7:32am

Got it!

My mistake was I thought I'd have to somehow change the SaveValues and LoadValues functions to work with arrays. That turned out to be unnecessary. The attached project saves and loads a 3-field structured class array. And it even has a generate button to automatically populate fields, if you don't want to manually enter data into each textbox. The code would be more streamlined if I could remember how to make textboxes into arrays, lol. Tunnel vision. :roll:

Great! So, the next step is to implement this code into a Task/Fork process to make it much more useful. I'll put that in my other thread. But it will have to wait until tomorrow. I'm out of coding time for today. :|

I hope you find this code useful.
Attachments
Object_Serialization_Array.tar.gz
(16.46 KiB) Downloaded 5 times

User avatar
cogier
Site Admin
Posts: 322
Joined: Wednesday 21st September 2016 2:22pm
Location: Guernsey, Channel Islands

Re: Object serialization, allowing save and load of class structure data

Post by cogier » Tuesday 08th October 2019 2:37pm

I looked up 'Object serialization' but I don't understand what it does. However you could simplify your code in places. Have a look at the attached.
Action.tar.gz
(14.66 KiB) Downloaded 2 times
The code would be more streamlined if I could remember how to make textboxes into arrays, lol. Tunnel vision. :roll:
Try this: -
Dim TextBoxes As TextBox[] = [TextBox1, TextBox2, TextBox3, TextBox4]

User avatar
Godzilla
Posts: 35
Joined: Wednesday 26th September 2018 11:20am

Re: Object serialization, allowing save and load of class structure data

Post by Godzilla » Wednesday 09th October 2019 8:07am

Hey cogier,

Thanks for your reply. Once again, I neglected to take the time to explain what my project actually does. Kicking myself to not do that again.

In layman's terms, object serialization is basically File.Save and File.Load on steroids. Its a way of saving and loading of complex structures, instead of a single variable type.

The Object_Serialization_Array project demonstrates how save and load an array of a class variable. The class variable I designed has 5 fields, of various data types. In theory, it could be designed to have hundreds or even thousands of fields, of all data types. And its all neatly packed in just one super variable, and all saved and loaded by this method into one single speedy little binary file. I love it.

So i made an array out of this super variable, of 3 indexes [0] [1] [2]. There's 15 textboxes where data can be entered into these fields. The first column of 5 textboxes is for array index [0]. The 2nd column is array index [1]. And the 3rd column is array index [2].

Entering data into these textboxes (or pressing the Generate button), you can then click the Save button and all the array indexes, along with all the different variable types, gets saved into one single binary file. You can close the program, run it again, and click the Load button. And everything that had been saved earlier is instantly loaded back into a perfectly re-created class array, from which all its fields and indexes appear back into their respective textboxes, .

The ability to save and load complex things opens up a world of flexibility and possibilities, when sending data from a Fork back to its parent program, before the Fork vanishes with all the work its done.

Your way of filling textboxes in the project you provided, I must say, very clever! I didn't know about the Action property. I like it. I'm currently retooling my Object_Serialization_Array project to use your Textboxes array method, which I should have learned from your other code submissions. Its a much better way and I need rely on it much more.

My 100% working Task_Test in my other thread will have to be delayed yet again. But there's not enough hours in the day to do the things I love, sigh. Weekend can't get here soon enough!

And lastly, some object serialization background, for those interested:
Serialization is the process of converting an object into a stream of bytes in order to store the object or transmit it to memory, a database, or a file. Its main purpose is to save the state of an object in order to be able to recreate it when needed. The reverse process is called deserialization.
continued here https://docs.microsoft.com/en-us/dotnet ... alization/

User avatar
Godzilla
Posts: 35
Joined: Wednesday 26th September 2018 11:20am

Re: Object serialization, allowing save and load of class structure data

Post by Godzilla » Friday 11th October 2019 7:20am

For the sake of completeness, I've taken Jussi Lahtinen's object serialization code and completed it by adding support for all the variables that he hadn't taken the time to include. They are Single, Float, Variant, Object, and Pointer. So now it includes the entire spectrum of Gambas variables. I haven't strictly tested each and every newly-added variable. But everything appears to be correct, and should work as expected.

The purpose is to save and load objects. It uses compact and fast binary files.

How to use, according to Jussi's instructions:
'This will save the object to file.

Dim MyObject As MyClass
Dim hFile As File

hFile = Open "~/Desktop/testtest" For Create
SaveValues(hFile, MyObject)
Close #hFile


'This will load the object from file.

hFile = Open "~/Desktop/testtest" For Read
LoadValues(hFile, MyObject)
Close #hFile 
This is the method I came up with to save an array, in this case a structured class array. But just modify as needed:
  FileObject = Open "/tmp/A_SubCategory_ByDate" For Write Create 
    For TheCounter = 0 To A_SubCategory_ByDate.Max
      SaveValues(FileObject, A_SubCategory_ByDate[TheCounter]) 
    Next
  Close #FileObject
This is the method I came up with to load an array, in this case a structured class array. But just modify as needed:
    FileObject = Open "/tmp/A_SubCategory_ByDate" For Read
    Do While Not Eof(FileObject)
      A_SubCategory_ByDate.Resize(A_SubCategory_ByDate.Count + 1)
      A_SubCategory_ByDate[TheCounter] = New DataStructure
      LoadValues(FileObject, A_SubCategory_ByDate[TheCounter])
      TheCounter = TheCounter + 1
    Loop
    Close #FileObject 
And here's the two functions that do the magic:
Public Function SaveValues(hStream As Stream, hObject As Object)

  Dim hCls As Class = Object.Class(hObject)
  Dim sTmp As String  
  
   
  For Each sTmp In hCls.Symbols.Sort(gb.Binary)
    If hCls[sTmp].Kind = Class.Variable Then
  
      Select Case hCls[sTmp].Type
  
      Case "h" 'short
        Write #hStream, Object.GetProperty(hObject, sTmp) As Short
  
      Case "b" 'boolean
        Write #hStream, Object.GetProperty(hObject, sTmp) As Boolean
  
      Case "c" 'byte
        Write #hStream, Object.GetProperty(hObject, sTmp) As Byte
  
      Case "i" 'integer
        Write #hStream, Object.GetProperty(hObject, sTmp) As Integer
  
      Case "l" 'long
        Write #hStream, Object.GetProperty(hObject, sTmp) As Long
  
      Case "s" 'string
        Write #hStream, Object.GetProperty(hObject, sTmp) As String
        
      Case "d" 'date
        Write #hStream, Object.GetProperty(hObject, sTmp) As Date
        
      Case "f" 'float
        Write #hStream, Object.GetProperty(hObject, sTmp) As Float
        
      Case "o" 'object
        Write #hStream, Object.GetProperty(hObject, sTmp) As Object
        
      Case "p" 'pointer
        Write #hStream, Object.GetProperty(hObject, sTmp) As Pointer
        
      Case "g" 'single
        Write #hStream, Object.GetProperty(hObject, sTmp) As Single
        
      Case "v" 'variant
        Write #hStream, Object.GetProperty(hObject, sTmp) As Variant
  
      Case Else
        Print "Missing variable type: " & hCls[sTmp].Type
        Error.Raise("Error! Missing variable type definition.")
      End Select    
    Endif
  Next
  

End

Public Function LoadValues(hStream As Stream, hObject As Object)

  Dim hCls As Class = Object.Class(hObject)
  Dim sTmp As String
  
  Dim h As Short
  Dim b As Boolean
  Dim c As Byte
  Dim i As Integer
  Dim l As Long
  Dim s As String
  Dim d As Date
  Dim f As Float
  Dim o As Object
  Dim p As Pointer
  Dim g As Single
  Dim v As Variant
  
  For Each sTmp In hCls.Symbols.Sort(gb.Binary)
    If hCls[sTmp].Kind = Class.Variable Then
  
      Select Case hCls[sTmp].Type
  
      Case "h" 'short
        h = Read #hStream As Short
        Object.SetProperty(hObject, sTmp, h)
  
      Case "b" 'boolean
        b = Read #hStream As Boolean
        Object.SetProperty(hObject, sTmp, b)
  
      Case "c" 'byte
        c = Read #hStream As Byte
        Object.SetProperty(hObject, sTmp, c)
  
      Case "i" 'integer
        i = Read #hStream As Integer
        Object.SetProperty(hObject, sTmp, i)
  
      Case "l" 'long
        l = Read #hStream As Long
        Object.SetProperty(hObject, sTmp, l)
  
      Case "s" 'string
        s = Read #hStream As String
        Object.SetProperty(hObject, sTmp, s)
  
      Case "d" 'date
        d = Read #hStream As Date
        Object.SetProperty(hObject, sTmp, d)
        
      Case "f" 'float
        f = Read #hStream As Float
        Object.SetProperty(hObject, sTmp, f)
        
      Case "o" 'object
        o = Read #hStream As Object
        Object.SetProperty(hObject, sTmp, o)
        
      Case "p" 'pointer
        p = Read #hStream As Pointer
        Object.SetProperty(hObject, sTmp, p)
        
      Case "g" 'single
        g = Read #hStream As Single
        Object.SetProperty(hObject, sTmp, g)
        
      Case "v" 'variant
        v = Read #hStream As Variant
        Object.SetProperty(hObject, sTmp, v)
  
      Case Else
        Print "Missing variable type: " & hCls[sTmp].Type
        Error.Raise("Error! Missing variable type definition.")
      End Select
  
    Endif
  Next

End

Post Reply