As mentioned in the first part of this series, some functions in Python can be dangerous if you’re not aware of their risks. In this installment, we’ll cover deserializing data with pickle and yaml and information leakage.

Pickle and friends

Why it’s useful

pickle enables you to store state and Python objects to disk so that you can later restore them. Pickle can be useful for storing something that doesn’t quite need a database or for data that’s inherently temporary.

In the past, I’ve used pickle to support pause and resume functionality for large file transfers. I saved the progress to a pickle file and then, on resume, picked up where it left off and removed the pickle.

Why it’s dangerous

Pickle has the same weaknesses as exec and eval , which we covered in part 1. It enables users to craft input that executes arbitrary code on your machine. Sound familiar?

Other libraries and modules rely upon Pickle to do their thing as well, which makes them prone to the same risks. One of those is shelve, which is another module related to serializing Python objects.

Celery, a popular package used for sending messages to queues, used pickle by default for communication with its workers before version 3.0.18. If you’re using an older version of Celery, make sure you’re following the recommended security guidelines or upgrade.

Django, a popular Python web framework, used pickle before version 1.6 to store session information. There’s a scary warning in the Django docs about how that can go wrong.

A dangerous example

I’m going to use an example from Lincoln Loop’s Playing with Pickle Security and expand upon it. In our example, we will serialize a command to call the command-line utility ls and deserialize it with pickle.loads().

import os
import cPickle


# Exploit that we want the target to unpickle
class Exploit(object):
    def __reduce__(self):
        # Note: this will only list files in your directory.
        # It is a proof of concept.
        return (os.system, ('ls',))


def serialize_exploit():
    shellcode = cPickle.dumps(Exploit())
    return shellcode


def insecure_deserialize(exploit_code):
    cPickle.loads(exploit_code)


if __name__ == '__main__':
    shellcode = serialize_exploit()
    print('Yar, here be yer files.')
    insecure_deserialize(shellcode)

In this case, we only wanted to list the files in the directory using the ls command. We could have used almost any shell command.

What to use instead

You could use json to serialize data or, if you must, yaml. If you use yaml, please read the section below on why it has its own set of risks.

If you’re using Celery or Django, you should upgrade to a version that does not use pickle for serialization.

If you must use it…

Be careful with your input! Never trust a pickle that has gone over the network or come from someone else. It’s too easy to exploit.

Additional references

Loading YAMLs

Why it’s useful

YAML files offer another option for serializing and deserializing data. They are useful for storing configuration or other immutable values. I have used YAMLs to store configuration values for web applications, where the configuration differs depending upon the environment we’re deploying to (production vs staging, for example).

PyYAML does not live in the standard library but seems like the most popular way to parse YAMLs in Python.

Why it’s dangerous

The simplest way to load a YAML file is with yaml.load(). Unfortunately, yaml.load() is an unsafe operation that, you guessed it, enables maliciously crafted files to execute arbitrary code on the host machine.

A dangerous example

As with pickle, we’ll setup an example where we read the files in a directory on the host machine.

In exploit.yml:

your_files: !!python/object/apply:subprocess.check_output ['ls']

In a Python script (after perhaps running pip install pyyaml):

import yaml

with open('exploit.yml') as exploit_file:
    contents = yaml.load(exploit_file)
    your_files = contents['your_files'].splitlines()
    for your_file in your_files:
        print(your_file)

Again, we can provide many different commands to subprocess, including those that we discussed in part 1.

What to use instead

The yaml module has a safe way to load yaml files: yaml.safe_load(). I wish the package had the safe method as the default, rather than the dangerous one.

As Ned Batchelder says:

Why do serialization implementers do this? If you must extend the format with dangerous features, provide them in the non-obvious method. Provide a .load() method and a .dangerous_load() method instead. At least that way people would have to decide to do the dangerous thing.

If you must use it…

Use yaml.safe_load(). If you must use yaml.load() directly, then you should be careful about which files you load and trust.

Additional references

A few more dangers

I wanted to briefly touch on a few other things to keep in mind while writing Python code.

SQL Injection

SQL Injection is basically untrusted input meets your database. All the same risks that we talked about with untrusted input above also apply here.

As a quick example, here’s how someone could exploit this:

import sqlite3

def get_user_by_name(name, cursor):
    cursor.execute("SELECT * FROM users WHERE name = '%s'" % name)  # unsafe!


if __name__ == '__main__':
    conn = sqlite3.connect('example.db')
    cursor = conn.cursor()
    malicious_name = "Joe'; DROP TABLE users; --"
    get_user_by_name(malicious_name, conn)

If you ran this example against a real database, the malicious name would drop the user’s table. Not great.

Python provides a database binding for sqlite3 in the standard library and there’s a section in the Python docs where they talk about how to properly escape variables (which we do not do in the example). Otherwise, I’d recommend using an ORM, such as the one in Django or sqlalchemy.

Information Leakage

The print function and logging module are useful but potentially risky. Ideally, any log files that we write have their permissions configured to allow only sufficiently privileged users to read them. If anyone can read the log file, it’s easier for someone to access the logs when they shouldn’t be able to do so. If you must log sensitive information (must you?), be sure to protect it through access controls.

Thanks to @goodwillbits for recommending that I add this section.

In Conclusion

In this series, we’ve covered a few different ways in which Python functions can be dangerous. Python’s documentation is good about letting you know when it’s risky to use something but you have to know when and where to look. If you take nothing else away from this post, please remember to be careful when it comes to accepting untrusted input.

Discussion on Hacker News.