Page 1 of 1

Get a download file size before download (gb.net.curl)

Posted: Monday 1st February 2021 2:39pm
by BruceSteers
Hi all

I'm downloading an archive using the gb.net.curl HttpClient object
Like the code below...

Public hClient As HttpClient
Public hFile As File


Public Sub GetArchive()
  Dim sFork As String = "gambas"
  Dim sName As String = "gambas-stable.zip"
  Dim sZip As String = "https://gitlab.com/gambas/gambas/-/archive/stable" &/ sName
  If Message.Question("Download archive?", "Yes", "No") <> 1 Then Return
  FMain.MeEnabled(False)
  hClient = New HttpClient As "hClient"
  hClient.Timeout = 20
  hClient.URL = sZip
  hFile = Open Parent &/ File.Name(sZip) For Write Create
  hClient.Get()
  
End

Public Sub hClient_Read()
  Dim sSize As String, Data As Variant
  Dim sTemp As String[]
  Read #hClient, Data, -256

  Dim iGot As Integer = hClient.Downloaded
 
  If iGot < 1024 Then
    sSize = Str(iGot) & " Bytes"
  Else If iGot < (1024 * 1024) Then
    sSize = Str(iGot / 1024)
    If InStr(sSize, ".") Then sSize = sSize[0, InStr(sSize, ".") + 2]
    sSize &= " Kilobytes"
  Else 
    sSize = Str(iGot / (1024 * 1024))
    If InStr(sSize, ".") Then sSize = sSize[0, InStr(sSize, ".") + 2]
    sSize &= " Megabytes"
  Endif
  
  Print "Downloading: " & sSize & "\r"

  Write #hFile, Data
End

Public Sub hClient_Finished()
Wait 0.1

 If hClient Then hClient.Close()
 If hFile Then hFile.Close()
  Print ("\nDownload Complete.\n")
 Wait 0.1
 FMain.MeEnabled(True)
 FMain.tvComp.Input("cd '" & Parent & "' && unzip  -ou " & File.Name(hClient.URL) & "\n")
 End
In the _Read() event i can get the download size but i can't find a way to get the file size before downloading.
I tried the HttpClient.Head() method but the list of headers did not have Content.Length ?
A bit puzzled.

And help appreciated :)
Bruce

Re: Get a download file size before download (gb.net.curl)

Posted: Monday 1st February 2021 3:49pm
by cogier
I have tried a few things, but I can't get the result you are looking for. If it helps there is a 'Size' property in 'Download' but I can't get it to work.

Image

Re: Get a download file size before download (gb.net.curl)

Posted: Monday 1st February 2021 6:00pm
by BruceSteers
cogier wrote: Monday 1st February 2021 3:49pm I have tried a few things, but I can't get the result you are looking for. If it helps there is a 'Size' property in 'Download' but I can't get it to work.

Image
Thanks Charlie.
I'd suspect that Size property is just downloaded not total size.

Puzzling.

Re: Get a download file size before download (gb.net.curl)

Posted: Monday 1st February 2021 6:43pm
by PJBlack
found this

Code: Select all

Use the -I option to only retrieve the headers, and look for the “Content-Length” header. Add the -L option if necessary to follow redirects. For example:

$ curl -L -I https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-9.4.0-amd64-netinst.iso
HTTP/1.1 302 Found
Date: Mon, 18 Jun 2018 09:51:32 GMT
Server: Apache/2.4.29 (Unix)
Location: https://gensho.ftp.acc.umu.se/debian-cd/current/amd64/iso-cd/debian-9.4.0-amd64-netinst.iso
Cache-Control: max-age=300
Expires: Mon, 18 Jun 2018 09:56:32 GMT
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 200 OK
Date: Mon, 18 Jun 2018 09:51:32 GMT
Server: Apache/2.4.29 (Unix)
Last-Modified: Sat, 10 Mar 2018 11:56:52 GMT
Accept-Ranges: bytes
Content-Length: 305135616
Age: 228
Content-Type: application/x-iso9660-image
may it helps ...

Re: Get a download file size before download (gb.net.curl)

Posted: Monday 1st February 2021 7:19pm
by BruceSteers
PJBlack wrote: Monday 1st February 2021 6:43pm found this

Code: Select all

Use the -I option to only retrieve the headers, and look for the “Content-Length” header. Add the -L option if necessary to follow redirects. For example:

$ curl -L -I https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-9.4.0-amd64-netinst.iso
HTTP/1.1 302 Found
Date: Mon, 18 Jun 2018 09:51:32 GMT
Server: Apache/2.4.29 (Unix)
Location: https://gensho.ftp.acc.umu.se/debian-cd/current/amd64/iso-cd/debian-9.4.0-amd64-netinst.iso
Cache-Control: max-age=300
Expires: Mon, 18 Jun 2018 09:56:32 GMT
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 200 OK
Date: Mon, 18 Jun 2018 09:51:32 GMT
Server: Apache/2.4.29 (Unix)
Last-Modified: Sat, 10 Mar 2018 11:56:52 GMT
Accept-Ranges: bytes
Content-Length: 305135616
Age: 228
Content-Type: application/x-iso9660-image
may it helps ...
It help yes but not as i'd hoped.
using curl -I i read the headers for the file in question (the gambas source archive)
https://gitlab.com/gambas/gambas/-/arch ... master.zip
the header list i got was something i had also managed to view using httpclient but did not see any content-length info.

and Benoit says this on the M/l..
Benoit wrote: Content-Length is not a mandatory header. So if it is not present, you
can't know by advance the size of the download.
so looks to me like the file i am trying to get the size of is one of those that does not report it :(.

Sigh , oh well.

Thanks for the help guys :)
Bruce

Re: Get a download file size before download (gb.net.curl)

Posted: Monday 1st February 2021 7:37pm
by BruceSteers
Hey Charlie.
Just an observation.

Line 8 on my first post does not show as i wrote it.

written it is just
Dim sZip As String = "the/url/to/fie"

the post shows href= and class= stuff that should not be there.

Just a note m8. dunno if it anything to do with you or not.
Bruce.

Re: Get a download file size before download (gb.net.curl)

Posted: Tuesday 16th February 2021 11:31am
by cogier
Hi Bruce,

Not sure what the issue is/was, seems OK to me or have I missed something?
Dim sZip As String = "the/url/to/fie"

Re: Get a download file size before download (gb.net.curl)

Posted: Tuesday 16th February 2021 1:35pm
by BruceSteers
cogier wrote: Tuesday 16th February 2021 11:31am Hi Bruce,

Not sure what the issue is/was, seems OK to me or have I missed something?
Dim sZip As String = "the/url/to/fie"
No the first post, not the last where i typed a url so it could be seen as it is supposed to look.
the first where it is wrong.

Re: Get a download file size before download (gb.net.curl)

Posted: Tuesday 16th February 2021 1:57pm
by cogier
Ahh yes, we have run into this before. I have corrected your first post. You need to add this: -

Image

Re: Get a download file size before download (gb.net.curl)

Posted: Tuesday 16th February 2021 1:59pm
by BruceSteers
cogier wrote: Tuesday 16th February 2021 1:57pm Ahh yes, we have run into this before. I have corrected your first post. You need to add this: -

Image
aah thanks m8, will remember that in future :)