[Fall 2021] Step 3: Run Pysa on open source projects

[onionymous]

Overview

Let's run Pysa on some open source projects and see if we can catch vulnerabilities! As another outcome, let's also write script(s) to annotate codebases and help us get better Pysa results.

The main goals of this project is to:

1. Run Pysa on open source Python projects, and if we find issues - maybe we can report some previously unfound vulnerabilities and file CVEs
2. If we're not catching issues that we should be - why not?
   1. If it's because of a lack of coverage, we should add to the list coverage improvements for Pysa so we can catch these issues.
   2. If it's a typing issue - we should write a script that automatically adds type annotations to a codebase so Pysa will work better with it out-of-the-box.

Since there's a ton of open source projects out there, multiple people can be working on this and trying out different things. :)

Running Pysa on open source

Here's a list of some popular open-source projects that are in Python that you could try running on:

• https://github.com/freedomofpress/securedrop
• https://github.com/arXiv/arxiv-base
• https://github.com/certbot/certbot

Feel free to try out Pysa on any other projects you think might be interesting!

Running Pysa on intentionally vulnerable applications

Running on intentionally vulnerable applications can be helpful since we know that there are definitely vulnerabilities in there, and we should aim to catch all of them. Here's a good list to start (feel free to try other projects you come across, though!):

• https://github.com/anxolerd/dvpwa
• https://github.com/fportantier/vulpy
• https://github.com/stamparm/DSVW
• https://github.com/rojan-rijal/Pyrate

When running Pysa on these projects, you may start to get a sense of what issues we're missing and why. A lot of the time this is due to those projects simply being untyped, so we should think of ways to add typing annotations to make it possible to catch the vulnerabilities.

We want to translate some of these coverage improvement strategies listed in our wiki (https://pyre-check.org/docs/pysa-coverage/) into a script (or multiple scripts!). For example, for a Django project, maybe we could write a script that annotates all untyped request parameters in view functions with HttpRequest, and similarly for Flask projects. Feel free to explore any other options that you think might be helpful!

Alternatively, maybe we don't have coverage for certain libraries used - in that case feel free to file a coverage improvement task (or even take it on yourself!)

Miscellaneous

Please see #17 for what fellows have done in the past related to this project.

[abishekvashok]

Hey ran pysa on all three projects and here are the findings:

The steps I used to run pysa on each project are:

• clone the repo
• create and activate a virtual environement
• setup pysa with: pyre init-pysa
• when asked for requirements file, I provided the path for one and then manually installed the requirements in the files in other folders.
• add python3.9/site-packages to search path in .pyre_configuration
• run pyre infer -I
• ran pyre analyze --no-verify

Secure drop

Triaged all reported issues except the one who's path is * for some reason and all are false positives.

[
  {
    "line": 157,
    "column": 32,
    "stop_line": 158,
    "stop_column": 56,
    "path": "securedrop/journalist_app/admin.py",
    "code": 5018,
    "name": "Open redirect",
    "description":
      "Open redirect [5018]: Data from [UserControlled] source(s) may be used in an open redirect via [Redirect] sink(s)",
    "define": "securedrop.journalist_app.admin.make_blueprint.add_user"
  },
  {
    "line": 207,
    "column": 28,
    "stop_line": 207,
    "stop_column": 73,
    "path": "securedrop/journalist_app/admin.py",
    "code": 5018,
    "name": "Open redirect",
    "description":
      "Open redirect [5018]: Data from [UserControlled] source(s) may be used in an open redirect via [Redirect] sink(s)",
    "define":
      "securedrop.journalist_app.admin.make_blueprint.reset_two_factor_hotp"
  },
  {
    "line": 195,
    "column": 24,
    "stop_line": 195,
    "stop_column": 69,
    "path": "securedrop/journalist_app/admin.py",
    "code": 5018,
    "name": "Open redirect",
    "description":
      "Open redirect [5018]: Data from [UserControlled] source(s) may be used in an open redirect via [Redirect] sink(s)",
    "define":
      "securedrop.journalist_app.admin.make_blueprint.reset_two_factor_totp"
  },
  {
    "line": 208,
    "column": 25,
    "stop_line": 208,
    "stop_column": 38,
    "path": "securedrop/source_app/main.py",
    "code": 5008,
    "name": "XSS",
    "description":
      "XSS [5008]: Data from [UserControlled] source(s) may reach [XSS] sink(s)",
    "define": "securedrop.source_app.main.make_blueprint.submit"
  },
  {
    "line": 55,
    "column": 41,
    "stop_line": 55,
    "stop_column": 49,
    "path": "*",
    "code": 5041,
    "name": "Possible shell injection via command line arguments",
    "description":
      "Possible shell injection via command line arguments [5041]: Data from [CLIUserControlled] source(s) may reach [RemoteCodeExecution] sink(s)",
    "define": "mako.cmd.cmdline"
  }
]

Recommendations: to prevent these false positives would be annotate render_template as a sanitiser for user controlled values..

Arxive Base

Couldn't detect any issues :/

[abishekvashok]

Certbot

Got issues. Triagged them, a hex function for some reason served as a sanitiser that rescued from ssrf. Hence, all issues reported are false positives except those with path "*".

[
  {
    "line": 532,
    "column": 36,
    "stop_line": 532,
    "stop_column": 44,
    "path": "*",
    "code": 5018,
    "name": "Open redirect",
    "description":
      "Open redirect [5018]: Data from [UserControlled] source(s) may be used in an open redirect via [Redirect] sink(s)",
    "define":
      "oauth2client.contrib.flask_util.UserOAuth2.required.curry_wrapper.required_wrapper"
  },
  {
    "line": 206,
    "column": 28,
    "stop_line": 206,
    "stop_column": 39,
    "path": "*",
    "code": 6066,
    "name": "Unsafe deserialization may result in RCE",
    "description":
      "Unsafe deserialization may result in RCE [6066]: Data from [UserControlled] source(s) may reach [ExecDeserializationSink] sink(s)",
    "define": "oauth2client.contrib.flask_util._get_flow_for_token"
  },
  {
    "line": 276,
    "column": 41,
    "stop_line": 276,
    "stop_column": 53,
    "path": "*",
    "code": 5041,
    "name": "Possible shell injection via command line arguments",
    "description":
      "Possible shell injection via command line arguments [5041]: Data from [CLIUserControlled] source(s) may reach [RemoteCodeExecution] sink(s)",
    "define": "sphinx.cmd.build.build_main"
  },
  {
    "line": 35,
    "column": 32,
    "stop_line": 36,
    "stop_column": 90,
    "path":
      "certbot-ci/certbot_integration_tests/utils/pebble_ocsp_server.py",
    "code": 5012,
    "name": "Potential Server-side request forgery (SSRF)",
    "description":
      "Potential Server-side request forgery (SSRF) [5012]: Data from [UserControlled] source(s) may reach [RequestSend_URI] sink(s)",
    "define":
      "certbot-ci.certbot_integration_tests.utils.pebble_ocsp_server._ProxyHandler.do_POST"
  },
  {
    "line": 36,
    "column": 39,
    "stop_line": 36,
    "stop_column": 70,
    "path":
      "certbot-ci/certbot_integration_tests/utils/pebble_ocsp_server.py",
    "code": 5010,
    "name": "User data to getattr",
    "description":
      "User data to getattr [5010]: Attacker may control at least one argument to getattr(,).",
    "define":
      "certbot-ci.certbot_integration_tests.utils.pebble_ocsp_server._ProxyHandler.do_POST"
  },
  {
    "line": 432,
    "column": 24,
    "stop_line": 432,
    "stop_column": 34,
    "path": "*",
    "code": 5018,
    "name": "Open redirect",
    "description":
      "Open redirect [5018]: Data from [UserControlled] source(s) may be used in an open redirect via [Redirect] sink(s)",
    "define": "oauth2client.contrib.flask_util.UserOAuth2.callback_view"
  },
  {
    "line": 413,
    "column": 35,
    "stop_line": 413,
    "stop_column": 46,
    "path": "*",
    "code": 6066,
    "name": "Unsafe deserialization may result in RCE",
    "description":
      "Unsafe deserialization may result in RCE [6066]: Data from [UserControlled] source(s) may reach [ExecDeserializationSink] sink(s)",
    "define": "oauth2client.contrib.flask_util.UserOAuth2.callback_view"
  }
]