Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add API endpoints for departments #1138

Open
wants to merge 87 commits into
base: develop
Choose a base branch
from
Open

Add API endpoints for departments #1138

wants to merge 87 commits into from

Conversation

michplunkett
Copy link
Collaborator

@michplunkett michplunkett commented Dec 18, 2024

Fixes issue

#385

Description of Changes

Add the baseline features for a data-focused API.

Tests and Linting

  • This branch is up-to-date with the develop branch.
  • pytest passes on my local development environment.
  • pre-commit passes on my local development environment.

@michplunkett michplunkett self-assigned this Dec 18, 2024
@michplunkett michplunkett linked an issue Dec 18, 2024 that may be closed by this pull request
@michplunkett michplunkett changed the title Add data API Add v1 data API Dec 19, 2024
@michplunkett
Copy link
Collaborator Author

Things that might be needed, are documentation for these endpoints. I'm not sure what a good protocol is for that rn, though I'm welcome to suggestions.

Comment on lines +38 to +41
resp_assignments = client.get(
url_for("v1.get_dept_assignments", department_id=department_id),
follow_redirects=True,
)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could probably get rid of follow_redirects, but I think it might be worth keeping in case we modify the endpoint at some point.

Comment on lines 709 to 823
"Location", backref=db.backref("incidents", cascade_backrefs=False)
"Location",
backref=db.backref("incidents", cascade_backrefs=False),
lazy="joined",
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addresses this periodic error: sqlalchemy.orm.exc.DetachedInstanceError: Parent instance <Incident at 0xffff731ba510> is not bound to a Session; lazy load operation of attribute 'address' cannot proceed (Background on this error at: h...

Comment on lines -267 to +383
officer = db.relationship("Officer", back_populates="descriptions")
id = db.Column(db.Integer, primary_key=True)
text_contents = db.Column(db.Text())
officer_id = db.Column(db.Integer, db.ForeignKey("officers.id", ondelete="CASCADE"))
officer = db.relationship("Officer", back_populates="descriptions")
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ordering for ease of reading.

Copy link
Collaborator

@abandoned-prototype abandoned-prototype left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, this is great work. I like the sharing of methods between csv download and API, that's great.

However, I do think that json output does not have the same limitations that csv has and we should make use of that. What I have in mind here is a response to /departments/<id>/officers that looks something like this:

[
  {
    "birth_year": 1967,
    "department_id": 3,
    "employment_date": "1993-04-12",
    "first_name": "MICHAEL",
    "gender": "M",
    "id": 32892,
    "last_name": "TYMOCH",
    "middle_initial": "K",
    "race": "WHITE",
    "suffix": null,
    "unique_internal_identifier": null,
    "assignments": [
      {
        "id": 15001,
        "job_id": 20,
        "officer_id": 32892,
        "resign_date": null,
        "star_no": "",
        "start_date": null,
        "unit_id": null
      },
      {
        "id": 88142,
        "job_id": 20,
        "officer_id": 32892,
        "resign_date": null,
        "star_no": "P3662",
        "start_date": null,
        "unit_id": 15
      },
      {
        "id": 88143,
        "job_id": 20,
        "officer_id": 32892,
        "resign_date": null,
        "star_no": "P2107",
        "start_date": null,
        "unit_id": 14
      }
    ],
    "salaries": [
      {
        "id": 15001,
        "is_fiscal_year": true,
        "officer_id": 32892,
        "overtime_pay": null,
        "salary": 63342,
        "year": 2020
      },
      {
        "id": 15002,
        "is_fiscal_year": true,
        "officer_id": 32892,
        "overtime_pay": null,
        "salary": 60042,
        "year": 2020
      },
      {
        "id": 15003,
        "is_fiscal_year": true,
        "officer_id": 32892,
        "overtime_pay": null,
        "salary": 50141,
        "year": 2020
      }
    ]
  },
  {
    "birth_year": 1995,
    "department_id": 3,
    "employment_date": "2018-06-18",
    "first_name": "BRITTANY",
    "gender": "F",
    "id": 32763,
    "last_name": "SANDS",
    "middle_initial": "E",
    "race": "WHITE",
    "suffix": null,
    "unique_internal_identifier": null,
    "assignments": [],
    "salaries": []
  },
  {
    "birth_year": 1958,
    "department_id": 3,
    "employment_date": "2014-08-25",
    "first_name": "ANDREW",
    "gender": "M",
    "id": 32607,
    "last_name": "NEWMARK",
    "middle_initial": "D",
    "race": "WHITE",
    "suffix": null,
    "unique_internal_identifier": null,
    "assignments": [],
    "salaries": []
  }
]

Basically all department data combined and reducing the number of endpoints to /officers and /incidents. I think that makes working with the data much easier, as otherwise one would have to call multiple different endpoints and then merge all the datasets.

It might make sense to omit null value entries, not sure.

I am also not sure how computationally expensive creating this output would be, so it might be useful (or necessary) to also include pagination to the endpoint output to alleviate that issue.
Similarly eventually we also want to support filters, either of these points could be a follow up.

In terms of documentation, I suggested swagger in the original issue. I haven't used it in a while so there might be better solutions out there by now

OpenOversight/app/models/database.py Outdated Show resolved Hide resolved
@michplunkett
Copy link
Collaborator Author

I think the Swagger point could be addressed now, but I'd like to address the nesting part in another PR. It's already quite large and the nesting part is a non-trivial addition, imo.

@@ -620,6 +591,211 @@ class Image(BaseModel, TrackUpdates):
)


class Incident(BaseModel, TrackUpdates):
Copy link
Collaborator Author

@michplunkett michplunkett Jan 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The classes were reordered so that I could accurately label return values for the Department class.

Comment on lines 858 to +859
__tablename__ = "users"

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This spacing is consistent with every other BaseModel class.

@michplunkett
Copy link
Collaborator Author

Tbh, I think it'd be worth adding the Swagger documentation in another PR and then we can deploy this once that's done. I'd rather make things more succinct than otherwise.

@michplunkett michplunkett changed the title Add API endpoints for departments Add first phase of API endpoints for departments Jan 24, 2025
@michplunkett michplunkett changed the title Add first phase of API endpoints for departments Add API endpoints for departments Jan 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants