Get a download file size before download (gb.net.curl)

For questions about Gambas web tools.
Post Reply
User avatar
BruceSteers
Posts: 521
Joined: Thursday 23rd July 2020 5:20pm

Get a download file size before download (gb.net.curl)

Post by BruceSteers » Monday 01st February 2021 2:39pm

Hi all

I'm downloading an archive using the gb.net.curl HttpClient object
Like the code below...

Public hClient As HttpClient
Public hFile As File


Public Sub GetArchive()
  Dim sFork As String = "gambas"
  Dim sName As String = "gambas-stable.zip"
  Dim sZip As String = "https://gitlab.com/gambas/gambas/-/archive/stable" &/ sName
  If Message.Question("Download archive?", "Yes", "No") <> 1 Then Return
  FMain.MeEnabled(False)
  hClient = New HttpClient As "hClient"
  hClient.Timeout = 20
  hClient.URL = sZip
  hFile = Open Parent &/ File.Name(sZip) For Write Create
  hClient.Get()
  
End

Public Sub hClient_Read()
  Dim sSize As String, Data As Variant
  Dim sTemp As String[]
  Read #hClient, Data, -256

  Dim iGot As Integer = hClient.Downloaded
 
  If iGot < 1024 Then
    sSize = Str(iGot) & " Bytes"
  Else If iGot < (1024 * 1024) Then
    sSize = Str(iGot / 1024)
    If InStr(sSize, ".") Then sSize = sSize[0, InStr(sSize, ".") + 2]
    sSize &= " Kilobytes"
  Else 
    sSize = Str(iGot / (1024 * 1024))
    If InStr(sSize, ".") Then sSize = sSize[0, InStr(sSize, ".") + 2]
    sSize &= " Megabytes"
  Endif
  
  Print "Downloading: " & sSize & "\r"

  Write #hFile, Data
End

Public Sub hClient_Finished()
Wait 0.1

 If hClient Then hClient.Close()
 If hFile Then hFile.Close()
  Print ("\nDownload Complete.\n")
 Wait 0.1
 FMain.MeEnabled(True)
 FMain.tvComp.Input("cd '" & Parent & "' && unzip  -ou " & File.Name(hClient.URL) & "\n")
 End
In the _Read() event i can get the download size but i can't find a way to get the file size before downloading.
I tried the HttpClient.Head() method but the list of headers did not have Content.Length ?
A bit puzzled.

And help appreciated :)
Bruce
Wishing well
Bruce

If at first you don't succeed , try it differently.

User avatar
cogier
Site Admin
Posts: 721
Joined: Wednesday 21st September 2016 2:22pm
Location: Guernsey, Channel Islands

Re: Get a download file size before download (gb.net.curl)

Post by cogier » Monday 01st February 2021 3:49pm

I have tried a few things, but I can't get the result you are looking for. If it helps there is a 'Size' property in 'Download' but I can't get it to work.

Image

User avatar
BruceSteers
Posts: 521
Joined: Thursday 23rd July 2020 5:20pm

Re: Get a download file size before download (gb.net.curl)

Post by BruceSteers » Monday 01st February 2021 6:00pm

cogier wrote:
Monday 01st February 2021 3:49pm
I have tried a few things, but I can't get the result you are looking for. If it helps there is a 'Size' property in 'Download' but I can't get it to work.

Image
Thanks Charlie.
I'd suspect that Size property is just downloaded not total size.

Puzzling.
Wishing well
Bruce

If at first you don't succeed , try it differently.

User avatar
PJBlack
Posts: 139
Joined: Tuesday 09th June 2020 10:26pm
Location: Florstadt, Hessen, Germany

Re: Get a download file size before download (gb.net.curl)

Post by PJBlack » Monday 01st February 2021 6:43pm

found this

Code: Select all

Use the -I option to only retrieve the headers, and look for the “Content-Length” header. Add the -L option if necessary to follow redirects. For example:

$ curl -L -I https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-9.4.0-amd64-netinst.iso
HTTP/1.1 302 Found
Date: Mon, 18 Jun 2018 09:51:32 GMT
Server: Apache/2.4.29 (Unix)
Location: https://gensho.ftp.acc.umu.se/debian-cd/current/amd64/iso-cd/debian-9.4.0-amd64-netinst.iso
Cache-Control: max-age=300
Expires: Mon, 18 Jun 2018 09:56:32 GMT
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 200 OK
Date: Mon, 18 Jun 2018 09:51:32 GMT
Server: Apache/2.4.29 (Unix)
Last-Modified: Sat, 10 Mar 2018 11:56:52 GMT
Accept-Ranges: bytes
Content-Length: 305135616
Age: 228
Content-Type: application/x-iso9660-image
may it helps ...

User avatar
BruceSteers
Posts: 521
Joined: Thursday 23rd July 2020 5:20pm

Re: Get a download file size before download (gb.net.curl)

Post by BruceSteers » Monday 01st February 2021 7:19pm

PJBlack wrote:
Monday 01st February 2021 6:43pm
found this

Code: Select all

Use the -I option to only retrieve the headers, and look for the “Content-Length” header. Add the -L option if necessary to follow redirects. For example:

$ curl -L -I https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-9.4.0-amd64-netinst.iso
HTTP/1.1 302 Found
Date: Mon, 18 Jun 2018 09:51:32 GMT
Server: Apache/2.4.29 (Unix)
Location: https://gensho.ftp.acc.umu.se/debian-cd/current/amd64/iso-cd/debian-9.4.0-amd64-netinst.iso
Cache-Control: max-age=300
Expires: Mon, 18 Jun 2018 09:56:32 GMT
Content-Type: text/html; charset=iso-8859-1

HTTP/1.1 200 OK
Date: Mon, 18 Jun 2018 09:51:32 GMT
Server: Apache/2.4.29 (Unix)
Last-Modified: Sat, 10 Mar 2018 11:56:52 GMT
Accept-Ranges: bytes
Content-Length: 305135616
Age: 228
Content-Type: application/x-iso9660-image
may it helps ...
It help yes but not as i'd hoped.
using curl -I i read the headers for the file in question (the gambas source archive)
https://gitlab.com/gambas/gambas/-/arch ... master.zip
the header list i got was something i had also managed to view using httpclient but did not see any content-length info.

and Benoit says this on the M/l..
Benoit wrote: Content-Length is not a mandatory header. So if it is not present, you
can't know by advance the size of the download.
so looks to me like the file i am trying to get the size of is one of those that does not report it :(.

Sigh , oh well.

Thanks for the help guys :)
Bruce
Wishing well
Bruce

If at first you don't succeed , try it differently.

User avatar
BruceSteers
Posts: 521
Joined: Thursday 23rd July 2020 5:20pm

Re: Get a download file size before download (gb.net.curl)

Post by BruceSteers » Monday 01st February 2021 7:37pm

Hey Charlie.
Just an observation.

Line 8 on my first post does not show as i wrote it.

written it is just
Dim sZip As String = "the/url/to/fie"

the post shows href= and class= stuff that should not be there.

Just a note m8. dunno if it anything to do with you or not.
Bruce.
Wishing well
Bruce

If at first you don't succeed , try it differently.

User avatar
cogier
Site Admin
Posts: 721
Joined: Wednesday 21st September 2016 2:22pm
Location: Guernsey, Channel Islands

Re: Get a download file size before download (gb.net.curl)

Post by cogier » Tuesday 16th February 2021 11:31am

Hi Bruce,

Not sure what the issue is/was, seems OK to me or have I missed something?
Dim sZip As String = "the/url/to/fie"

User avatar
BruceSteers
Posts: 521
Joined: Thursday 23rd July 2020 5:20pm

Re: Get a download file size before download (gb.net.curl)

Post by BruceSteers » Tuesday 16th February 2021 1:35pm

cogier wrote:
Tuesday 16th February 2021 11:31am
Hi Bruce,

Not sure what the issue is/was, seems OK to me or have I missed something?
Dim sZip As String = "the/url/to/fie"
No the first post, not the last where i typed a url so it could be seen as it is supposed to look.
the first where it is wrong.
Wishing well
Bruce

If at first you don't succeed , try it differently.

User avatar
cogier
Site Admin
Posts: 721
Joined: Wednesday 21st September 2016 2:22pm
Location: Guernsey, Channel Islands

Re: Get a download file size before download (gb.net.curl)

Post by cogier » Tuesday 16th February 2021 1:57pm

Ahh yes, we have run into this before. I have corrected your first post. You need to add this: -

Image

User avatar
BruceSteers
Posts: 521
Joined: Thursday 23rd July 2020 5:20pm

Re: Get a download file size before download (gb.net.curl)

Post by BruceSteers » Tuesday 16th February 2021 1:59pm

cogier wrote:
Tuesday 16th February 2021 1:57pm
Ahh yes, we have run into this before. I have corrected your first post. You need to add this: -

Image
aah thanks m8, will remember that in future :)
Wishing well
Bruce

If at first you don't succeed , try it differently.

Post Reply