Source

What is Pickle in python?

Hi there fellas. In this post i am going to tell you about pickle. It is used for serializing and de-serializing a Python object structure. Any object in python can be pickled so that it can be saved on disk. What pickle does is that it “serialises” the object first before writing it to file. Pickling is a way to convert a python object (list, dict, etc.) into a character stream. The idea is that this character stream contains all the information necessary to reconstruct the object in another python script. So lets continue:

  • First of all you have to import it through this command:
import pickle

pickle has two main methods. The first one is dump, which dumps an object to a file object and the second one is load, which loads an object from a file object.

  • Prepare something to pickle:

Now you just have to get some data which you can pickle. For the sake of simplicity i will be pickling a python list. So just read the below code and you will be able to figure it out yourself.

import pickle

a = ['test value','test value 2','test value 3']
a
# ['test value','test value 2','test value 3']

file_Name = "testfile"
# open the file for writing
fileObject = open(file_Name,'wb') 

# this writes the object a to the
# file named 'testfile'
pickle.dump(a,fileObject)   

# here we close the fileObject
fileObject.close()
# we open the file for reading
fileObject = open(file_Name,'r')  
# load the object from the file into var b
b = pickle.load(fileObject)  
b
# ['test value','test value 2','test value 3']
a==b
# True

The above code is self explanatory. It shows you how to import the pickled object and assign it to a variable. So now we need to know where we should use pickling. It is very useful when you want to dump some object while coding in the python shell. So after dumping whenever you restart the python shell you can import the pickled object and deserialize it. But there are other use cases as well which i found on stackoverflow. Let me list them below.

  1. saving a program’s state data to disk so that it can carry on where it left off when restarted (persistence)`
  2. sending python data over a TCP connection in a multi-core or distributed system (marshalling)
  3. storing python objects in a database
  4. converting an arbitrary python object to a string so that it can be used as a dictionary key (e.g. for caching & memoization).

One thing to note is that there is a brother of pickle as well with the name of cpickle. As the name suggests it is written in c which makes it 1000 times more faster than pickle. So why should we ever use pickle instead of cpickle? Here’s the reason:

  • pickle handles unicode objects
  • pickle is written in pure Python, it’s easier to debug.

For further reading i suggest the official pickle documentation or if you want to read more tutorials then check out the sqlite tutorial. Now we come to the end of today’s post. I hope you liked it. Do follow my blog to get regular updates. If you have any suggestions or comments then post them below.

Newsletter

×

If you liked what you read then I am sure you will enjoy a newsletter of the content I create. I am also publishing my "Practical Python Projects" book soon. To stay updated regarding the book and new posts, join 5000+ other people who receive my newsletter:

I send out the newsletter once every couple of weeks. No spam, I promise

✍️ Comments

Rahul

Thanks, for the pickle example

Rishit

Nice explanation, very precise !

redpeas

Reblogged this on Nate's XBRL Blog and commented: A nice summary of Pickling

igarcia

Thank’s a lot! I’m using ‘pickle’ to identify unussual AS numbers in an IP networks.

benjsec S

In the example ‘a’ is a list, not a dictionary as stated in the explanatory text.

Yasoob
In reply to benjsec S

Hey there! Sorry for not correcting it before. I have corrected it now. I am not sure how that typo slipped through. Thank you very much for letting me know. :)

Nani

Good

Ja'Crispy

This is more a tutorial on how to pickle that what pickling is and it’s workings in python.

felsen

Really very nice explanation….

Manish

Good post than I found on stackoverflow, very precise and to the point

garthj88

Great, thanks for answering my query.

Snigdha Batra

Really to the point article. As pointed earlier also, please update the text to mention the example is of a list , not a dictionary.

Yasoob
In reply to Snigdha Batra

Thanks for letting me know. I don’t know how that typo slipped through and why I didn’t correct it before. :) My bad.

D Roberts

This helped me understand pickle in Python a tad bit more, but I would have liked to see a real-world example to understand it is used (as where you mentioned storing a state).

D Roberts
In reply to D Roberts

rather “understand how it is used”….

Vireont

Thanks for step-by-step tutorial!

Tarun Kumar

Thanks. This was a good introduction to pickle.

Jeff Loughridge

Yasoob, nice work! I ran your example on python3 and wanted to point out to your readers a minor change for python3 compatibility. Python3 requires an encoding parameter to the open method for reading binary files. The line “b = pickle.load(fileObject)” will raise an exception unless “fileObject = open(file_Name,‘r’)” is changed to “fileObject = open(file_Name,‘rb’).

Yasoob
In reply to Jeff Loughridge

Hey Jeff! Thanks for pointing this out. It’s been a long time since I wrote this post so thank you very much for correcting it :)

Gurpartap Singh
In reply to Jeff Loughridge

yes I was going to raise the same point. Great Example BTW

Will

Thanks for the explanation!

khaledsalahblog

that’s helpful , thanks :)

Sri Ravi Devineni

very nice explanation

Sheeraz

Thanks for Simple and yet clear explanation!!

Saurav Jha

While loading the pickled object back, the file should be opened in “rb” mode instead of just ‘r’ since we have dumped initially by writing in raw binary “wb” mode. If not done, this throws the error (UnicodeDecodeError: ‘utf-8’ codec can’t decode byte 0x80 in position 0: invalid start byte).

asd

My friend, thanks for sharing. However if u load pickle u need to open file as ‘rb’ mode also, otherwise it will not work. So add to: fileObject = open(file_Name,‘r’) —> fileObject = open(file_Name,‘rb’) [Notice ‘rb’ in mode] :) You explained it well, good job. Take care

WuPeii

Very clear to understand, thanks alot.

anjankumar k

nice explanation , getting clearly first attempt itselft..hanks yasoob

Archito

Really helpful !!!

Jaison

in Python3 use ‘rb’ and not just ‘r’ when opening file

Say something

Send me an email when someone comments on this post.

Thank you!

Your comment has been submitted and will be published once it has been approved. 😊

OK