Gh metrics

[dioptx]

Contents

Experimentation Notebooks (Self contained)

• Defillama Specific notebooks
• PoC for Github Metrics

Functionality updates

• Ported new Github metric functionality in github_platform_metrics.py

Overview for github_platform_metrics.py

This file fetches and processes GitHub repository metrics

Key Components

• Rate Limit Handling: Functions manage GitHub API rate limits, pausing execution when limits are reached.
• GitHubFetcher: Fetches data from GitHub. Retrieves additional metadata for pull requests, such as reviews and comments.
• Pull Request Processing: Calculates metrics like time-to-approval and time-to-merge for each PR

Main Functions

• pull_github_data: Fetches data and returns a GitHubData object.
• pull_and_write_platforms_github_metrics: Writes DataFrames to a data store for persistence.

Run Schedule

• Runs Daily for all repositories

Update Schedule

• All PRs that are not merged need to be checked every day for updates. Their metrics need to be updated daily.
• When a PR is approved & merged, all metrics for its lifecycle can be calculated so there's no need to keep checking it every day. We only need to do it once in the initial pull.
• [IMPORTANT] New comments on already merged PRs may occur but they are out of scope for this effort since no metrics rely on this information

[lithium323]

thanks so much for this head start work Dennis! I started looking at the branch and will push some commits to it with my changes. I want to keep the old github code that goes to bigquery so we can have some overlap between old and new as we migrate.