The urllib2 module can be used to download data from the web (network resource access). This data can be a file, a website or whatever you want Python to download. The module supports HTTP, HTTPS, FTP and several other protocols. What is Urllib2? Urllib2 is a Python module for fetching URLs. What can it do? It offers a very simple interface, in the form of the urlopen function. Urlopen is capable of fetching URLs using a variety of different protocols like (http, ftp, file). It also offers an interface for. Urllib2 extension which permit multipart file upload. Download files. Download the file for your platform. If you're not sure which to choose, learn more about installing packages. This opens up as many doors for your programs as the internet opens up for you. Urllib in Python 3 is slightly different than urllib2 in Python 2, but they are mostly the same. Through urllib, you can access websites, download data, parse data, modify your headers, and do any GET and POST requests you might need to do.
Active5 months ago
I am creating a program that will download a .jar (java) file from a web server, by reading the URL that is specified in the .jad file of the same game/application. I'm using Python 3.2.1
I've managed to extract the URL of the JAR file from the JAD file (every JAD file contains the URL to the JAR file), but as you may imagine, the extracted value is type() string.
Here's the relevant function:
However I always get an error saying that the type in the function above has to be bytes, and not string. I've tried using the URL.encode('utf-8'), and also bytes(URL,encoding='utf-8'), but I'd always get the same or similar error.
So basically my question is how to download a file from a server when the URL is stored in a string type?
Bo MilanovichBo Milanovich
3,53366 gold badges3636 silver badges5353 bronze badges
7 Answers
If you want to obtain the contents of a web page into a variable, just
read the response of urllib.request.urlopen :
The easiest way to download and save a file is to use the
urllib.request.urlretrieve function:
But keep in mind that
urlretrieve is considered legacy and might become deprecated (not sure why, though).
Python Urllib2 Download File Converter
So the most correct way to do this would be to use the
urllib.request.urlopen function to return a file-like object that represents an HTTP response and copy it to a real file using shutil.copyfileobj .
If this seems too complicated, you may want to go simpler and store the whole download in a
bytes object and then write it to a file. But this works well only for small files.
It is possible to extract Oleh PrypinOleh Prypin
.gz (and maybe other formats) compressed data on the fly, but such an operation probably requires the HTTP server to support random access to the file.
23.8k66 gold badges7272 silver badges8888 bronze badges
Python Urllib2 Download File Mac
I use
requests package whenever I want something related to HTTP requests because its API is very easy to start with:
first, install
requests
then the code:
Ali Faki
Python Urllib2 Download File FreeAli Faki
2,00133 gold badges1111 silver badges2121 bronze badges
I hope I understood the question right, which is: how to download a file from a server when the URL is stored in a string type?
I download files and save it locally using the below code:
Ranvijay KumarRanvijay Kumar
![]()
Here we can use urllib's Legacy interface in Python3:
The following functions and classes are ported from the Python 2 module urllib (as opposed to urllib2). They might become deprecated at some point in the future.
Example (2 lines code):
Yang YuYang Yu
You can use wget which is popular downloading shell tool for that. https://pypi.python.org/pypi/wgetThis will be the simplest method since it does not need to open up the destination file. Here is an example.
Lasith NiroshanLasith Niroshan
Yes, definietly requests is great package to use in something related to HTTP requests. but we need to be careful with the encoding type of the incoming data as well below is an example which explains the difference
KaushalKaushal
user7726287user7726287
protected by SheldoreJul 21 at 1:21
Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).
Would you like to answer one of these unanswered questions instead? Not the answer you're looking for? Browse other questions tagged pythonpython-3.x or ask your own question.
Active1 year, 8 months ago
I'm downloading an entire directory from a web server. It works OK, but I can't figure how to get the file size before download to compare if it was updated on the server or not. Can this be done as if I was downloading the file from a FTP server?
@Jon: thank for your quick answer. It works, but the filesize on the web server is slightly less than the filesize of the downloaded file.
Examples:
It has anything to do with the CR/LF conversion?
Prashant Kumar
12.5k1313 gold badges4141 silver badges6060 bronze badges
PabloGPabloG
20.1k1010 gold badges4040 silver badges5454 bronze badges
7 Answers
I have reproduced what you are seeing:
Outputs this:
code# Threading example import time, thread def myfunction(string, sleeptime, lock,.args): while True: lock.acquire time.sleep(sleeptime) lock.release time. Flow of execution python. In Python flow is sequential as long as successive statements are indented the same amount. The for statement introduces indented sub-statements after the for-loop heading. Flow of control is often easy to visualize and understand if we draw a flowchart. This flowchart shows the exact steps and logic of how the for statement executes. When you run a single Python script from the command-line with python script.py, interpretation starts at the first line and continues line by line. If a line starts a class or function definition, the definition is stored for later reference. If the line is executable code, it is directly executed.
What am I doing wrong here? Is os.stat().st_size not returning the correct size?
Edit:OK, I figured out what the problem was:
this outputs:
Open Download File
Make sure you are opening both files for binary read/write.
Jonathan WorksJonathan Works
1,62511 gold badge1515 silver badges1313 bronze badges
Using the returned-urllib-object method
info() , you can get various information on the retrived document. Example of grabbing the current Google logo:
It's a dict, so to get the size of the file, you do
urllibobject.info()['Content-Length']
The doubled workload slows production, which raises the cost of the final product by diminishing the workers' potential output per paid period of work. Probability of Errors. Parallel conversion means.
And to get the size of the local file (for comparison), you can use the os.stat() command:
dbrdbr
125k5858 gold badges258258 silver badges320320 bronze badges
The size of the file is sent as the Content-Length header. Here is how to get it with urllib:
Jonathan WorksJonathan Works
1,62511 gold badge1515 silver badges1313 bronze badges
Also if the server you are connecting to supports it, look at Etags and the If-Modified-Since and If-None-Match headers.
Using these will take advantage of the webserver's caching rules and will return a 304 Not Modified status code if the content hasn't changed.
Jonathan WorksJonathan Works
1,62511 gold badge1515 silver badges1313 bronze badges
A requests-based solution using HEAD instead of GET (also prints HTTP headers):
It acts as a plug-in for Adobe Premiere to import flash video(flv). Each file is catalogued and designed to be printed so that you can build your own reference book. Adobe gamma calibration download software. A thumbnail document for each package is also provided as well as 25 FREE bonus images are included. This collection is designed by and for cutting.Size: 54.5 MB License: Freeware Price: $50 Keywords: -Data Added 1: August 09, 2007Moyea FLV Importer is a very easy-to-use yet professional software tool. With it, you can use Adobe Premiere to process and edit flv files along with other audio & video files.
Usageccpizzaccpizza
14k88 gold badges9292 silver badges9696 bronze badges
MadhusudhanMadhusudhan
For a python3 (tested on 3.5) approach I'd recommend:
yukashima huksayyukashima huksay
1,40922 gold badges1515 silver badges3838 bronze badges
Not the answer you're looking for? Browse other questions tagged pythonurllib or ask your own question.Comments are closed.
|
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |