Get a download file size before download (gb.net.curl)

For questions about Gambas web tools.
Post Reply
User avatar
BruceSteers
Posts: 1523
Joined: Thursday 23rd July 2020 5:20pm
Location: Isle of Wight
Contact:

Get a download file size before download (gb.net.curl)

Post by BruceSteers »

Hi all

I'm downloading an archive using the gb.net.curl HttpClient object
Like the code below...

Public hClient As HttpClient
Public hFile As File


Public Sub GetArchive()
  Dim sFork As String = "gambas"
  Dim sName As String = "gambas-stable.zip"
  Dim sZip As String = "https://gitlab.com/gambas/gambas/-/archive/stable" &/ sName
  If Message.Question("Download archive?", "Yes", "No") <> 1 Then Return
  FMain.MeEnabled(False)
  hClient = New HttpClient As "hClient"
  hClient.Timeout = 20
  hClient.URL = sZip
  hFile = Open Parent &/ File.Name(sZip) For Write Create
  hClient.Get()
  
End

Public Sub hClient_Read()
  Dim sSize As String, Data As Variant
  Dim sTemp As String[]
  Read #hClient, Data, -256

  Dim iGot As Integer = hClient.Downloaded
 
  If iGot < 1024 Then
    sSize = Str(iGot) & " Bytes"
  Else If iGot < (1024 * 1024) Then
    sSize = Str(iGot / 1024)
    If InStr(sSize, ".") Then sSize = sSize[0, InStr(sSize, ".") + 2]
    sSize &= " Kilobytes"
  Else 
    sSize = Str(iGot / (1024 * 1024))
    If InStr(sSize, ".") Then sSize = sSize[0, InStr(sSize, ".") + 2]
    sSize &= " Megabytes"
  Endif
  
  Print "Downloading: " & sSize & "\r"

  Write #hFile, Data
End

Public Sub hClient_Finished()
Wait 0.1

 If hClient Then hClient.Close()
 If hFile Then hFile.Close()
  Print ("\nDownload Complete.\n")
 Wait 0.1
 FMain.MeEnabled(True)
 FMain.tvComp.Input("cd '" & Parent & "' && unzip  -ou " & File.Name(hClient.URL) & "\n")
 End
In the _Read() event i can get the download size but i can't find a way to get the file size before downloading.
I tried the HttpClient.Head() method but the list of headers did not have Content.Length ?
A bit puzzled.

And help appreciated :)
Bruce
If at first you don't succeed , try doing something differently.
BruceS
User avatar
cogier
Site Admin
Posts: 1118
Joined: Wednesday 21st September 2016 2:22pm
Location: Guernsey, Channel Islands

Re: Get a download file size before download (gb.net.curl)

Post by cogier »

I have tried a few things, but I can't get the result you are looking for. If it helps there is a 'Size' property in 'Download' but I can't get it to work.

Image
User avatar
BruceSteers
Posts: 1523
Joined: Thursday 23rd July 2020 5:20pm
Location: Isle of Wight
Contact:

Re: Get a download file size before download (gb.net.curl)

Post by BruceSteers »

cogier wrote: Monday 1st February 2021 3:49pm I have tried a few things, but I can't get the result you are looking for. If it helps there is a 'Size' property in 'Download' but I can't get it to work.

Image
Thanks Charlie.
I'd suspect that Size property is just downloaded not total size.

Puzzling.
If at first you don't succeed , try doing something differently.
BruceS
User avatar
PJBlack
Posts: 184
Joined: Tuesday 9th June 2020 10:26pm
Location: Florstadt, Hessen, Germany

Re: Get a download file size before download (gb.net.curl)

Post by PJBlack »

found this

Code: Select all

Use the -I option to only retrieve the headers, and look for the “Content-Length” header. Add the -L option if necessary to follow redirects. For example:

$ curl -L -I https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-9.4.0-amd64-netinst.iso
HTTP/1.1 302 Found
Date: Mon, 18 Jun 2018 09:51:32 GMT
Server: Apache/2.4.29 (Unix)
Location: https://gensho.ftp.acc.umu.se/debian-cd/current/amd64/iso-cd/debian-9.4.0-amd64-netinst.iso
Cache-Control: max-age=300
Expires: Mon, 18 Jun 2018 09:56:32 GMT
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 200 OK
Date: Mon, 18 Jun 2018 09:51:32 GMT
Server: Apache/2.4.29 (Unix)
Last-Modified: Sat, 10 Mar 2018 11:56:52 GMT
Accept-Ranges: bytes
Content-Length: 305135616
Age: 228
Content-Type: application/x-iso9660-image
may it helps ...
User avatar
BruceSteers
Posts: 1523
Joined: Thursday 23rd July 2020 5:20pm
Location: Isle of Wight
Contact:

Re: Get a download file size before download (gb.net.curl)

Post by BruceSteers »

PJBlack wrote: Monday 1st February 2021 6:43pm found this

Code: Select all

Use the -I option to only retrieve the headers, and look for the “Content-Length” header. Add the -L option if necessary to follow redirects. For example:

$ curl -L -I https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-9.4.0-amd64-netinst.iso
HTTP/1.1 302 Found
Date: Mon, 18 Jun 2018 09:51:32 GMT
Server: Apache/2.4.29 (Unix)
Location: https://gensho.ftp.acc.umu.se/debian-cd/current/amd64/iso-cd/debian-9.4.0-amd64-netinst.iso
Cache-Control: max-age=300
Expires: Mon, 18 Jun 2018 09:56:32 GMT
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 200 OK
Date: Mon, 18 Jun 2018 09:51:32 GMT
Server: Apache/2.4.29 (Unix)
Last-Modified: Sat, 10 Mar 2018 11:56:52 GMT
Accept-Ranges: bytes
Content-Length: 305135616
Age: 228
Content-Type: application/x-iso9660-image
may it helps ...
It help yes but not as i'd hoped.
using curl -I i read the headers for the file in question (the gambas source archive)
https://gitlab.com/gambas/gambas/-/arch ... master.zip
the header list i got was something i had also managed to view using httpclient but did not see any content-length info.

and Benoit says this on the M/l..
Benoit wrote: Content-Length is not a mandatory header. So if it is not present, you
can't know by advance the size of the download.
so looks to me like the file i am trying to get the size of is one of those that does not report it :(.

Sigh , oh well.

Thanks for the help guys :)
Bruce
If at first you don't succeed , try doing something differently.
BruceS
User avatar
BruceSteers
Posts: 1523
Joined: Thursday 23rd July 2020 5:20pm
Location: Isle of Wight
Contact:

Re: Get a download file size before download (gb.net.curl)

Post by BruceSteers »

Hey Charlie.
Just an observation.

Line 8 on my first post does not show as i wrote it.

written it is just
Dim sZip As String = "the/url/to/fie"

the post shows href= and class= stuff that should not be there.

Just a note m8. dunno if it anything to do with you or not.
Bruce.
If at first you don't succeed , try doing something differently.
BruceS
User avatar
cogier
Site Admin
Posts: 1118
Joined: Wednesday 21st September 2016 2:22pm
Location: Guernsey, Channel Islands

Re: Get a download file size before download (gb.net.curl)

Post by cogier »

Hi Bruce,

Not sure what the issue is/was, seems OK to me or have I missed something?
Dim sZip As String = "the/url/to/fie"
User avatar
BruceSteers
Posts: 1523
Joined: Thursday 23rd July 2020 5:20pm
Location: Isle of Wight
Contact:

Re: Get a download file size before download (gb.net.curl)

Post by BruceSteers »

cogier wrote: Tuesday 16th February 2021 11:31am Hi Bruce,

Not sure what the issue is/was, seems OK to me or have I missed something?
Dim sZip As String = "the/url/to/fie"
No the first post, not the last where i typed a url so it could be seen as it is supposed to look.
the first where it is wrong.
If at first you don't succeed , try doing something differently.
BruceS
User avatar
cogier
Site Admin
Posts: 1118
Joined: Wednesday 21st September 2016 2:22pm
Location: Guernsey, Channel Islands

Re: Get a download file size before download (gb.net.curl)

Post by cogier »

Ahh yes, we have run into this before. I have corrected your first post. You need to add this: -

Image
User avatar
BruceSteers
Posts: 1523
Joined: Thursday 23rd July 2020 5:20pm
Location: Isle of Wight
Contact:

Re: Get a download file size before download (gb.net.curl)

Post by BruceSteers »

cogier wrote: Tuesday 16th February 2021 1:57pm Ahh yes, we have run into this before. I have corrected your first post. You need to add this: -

Image
aah thanks m8, will remember that in future :)
If at first you don't succeed , try doing something differently.
BruceS
Post Reply