Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Adds draft design doc for new block explorer REST API endpoints #3342

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
160 changes: 160 additions & 0 deletions docs/design/additional-endpoints.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
# Design Document: ERC20/ERC721/ERC1155 Token Transfer Events API, Tokens Owned by an Address
konstantinabl marked this conversation as resolved.
Show resolved Hide resolved

## Overview
- This document outlines the design for implementing new endpoints allowing exposure of more EVM centric data, similar to Etherscan's and BlockScout's APIs.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- This document outlines the design for implementing new endpoints allowing exposure of more EVM centric data, similar to Etherscan's and BlockScout's APIs.
- This document outlines the design for implementing new endpoints allowing exposure of more EVM centric data that support Block Explorer (e.g. Etherscan, BlockScout) API needs.

+ The endpoints will support:
+ - ERC20 token transfers
+ - ERC721 NFT transfers
+ - ERC1155 multi-token transfers
+ - Tokens owned by an address

## Problem Statement
- Currently MN doesn't support EVM centric queries, like token transfer events for specific standards like ERC20, ERC721, ERC1155 or balance of an address for a specific token.

## Goals
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please take a look at
https://api-testnet.snowtrace.io/api?module=contract&action=getsourcecode&address=0x5425890298aed601595a70AB815c96711a31Bc65, a user may use this when looking for ABI
I think this might be another goal to capture. P2 in nature for this story though.
This is very interesting as it may require calling the verification service.

@acuarica do you see any other option to retrieve such information?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the verification service is the source of truth when getting the ABI.

- Provide API endpoints to fetch token transfer events
+ - Support filtering by sender address
+ - Support filtering by block range
+ - Support filtering by specific token contract
+ - Support ERC20, ERC721, and ERC1155 token standards
- Match Etherscan's response format for easy integration
- Maintain performance with large datasets
- Provide endpoints to fetch tokens owned by an address

Nana-EC marked this conversation as resolved.
Show resolved Hide resolved
## Proposed Solution

### Overview

Introduce two new endpoints:
quiet-node marked this conversation as resolved.
Show resolved Hide resolved
- `getTokenTransfers` - to fetch token transfer events
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: what are the params?
You should move the params explanation lower to here.
Also note what's mandatory

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So i think in the overview its enough to state the format of the endpoint and the detailed explanation down where it is, do you think thats fine or should I move it all to this section?
Also I have chosen this classic REST format, which differentiates from the way etherscan and blockscout has implemented their endpoints, entirely with query parameters, not sure if thats gonna affect the user. Do they expect the exactly same format
In etherscan the ednpoints look like this:

GET /api
   ?module=account
   &action=tokentx
   &contractaddress=0x9f8f72aa9304c8b593d555f12ef6589cc3a579a2
   &address=0x4e83362442b8d1bec281594cea3050c8eb01311c
   &page=1
   &offset=100
   &startblock=0
   &endblock=27025780
   &sort=asc
   &apikey=YourApiKeyToken

- `getTokensOwnedByAddress` - to fetch tokens owned by an address (TBD)

## `getTokenTransfers`

### Components
konstantinabl marked this conversation as resolved.
Show resolved Hide resolved

#### 1. EthImpl
Main service class handling token transfer event logic:
- Validates block ranges
- Filters logs for token transfer events
- Checks the contracts in the MN response for the token standard needed
- Transforms logs to transfer events format

#### 2. CommonService Integration
Utilizes existing CommonService for:
- Block range validation
- Timestamp parameter handling

#### 3. MirrorNodeClient Integration
Uses MirrorNodeClient for:
- Fetching contract logs
- Block information retrieval

### Data Flow
```mermaid
sequenceDiagram
Client->>EthImpl: getTokenTransfers()
EthImpl->>CommonService: getLogs()
CommonService->>CommonService: getLogsWithParams()
CommonService->>MirrorNodeClient: getContractResultsLogs()
MirrorNodeClient->>CommonService: ''
CommonService->>EthImpl: ''
quiet-node marked this conversation as resolved.
Show resolved Hide resolved
EthImpl->>EthImpl: getErcRegistry()
EthImpl->>EthImpl: checkContracts()
EthImpl->>Client: Return Response
```
quiet-node marked this conversation as resolved.
Show resolved Hide resolved

### Technical Details

#### API Interfaces

```typescript
interface TokenTransferEvent {
blockNumber: string;
timeStamp: string;
hash: string;
nonce: number;
blockHash: string;
from: string;
contractAddress: string;
to: string;
tokenId?: string;
tokenName?: string;
tokenSymbol?: string;
tokenDecimal?: number;
transactionIndex: number;
gas: number;
gasPrice: number;
gasUsed: number;
cumulativeGasUsed: number;
input: string;
confirmations: number;
}
```
quiet-node marked this conversation as resolved.
Show resolved Hide resolved

#### Key Implementation Details

The `getTokenTransfers` endpoint will be calling the same method in the `EthImpl` class which will accept the following parameters:
quiet-node marked this conversation as resolved.
Show resolved Hide resolved

- `address`: The address to filter transfers by
- `fromBlock`: The starting block number
- `toBlock`: The ending block number
- `contractAddress`: The address of the token contract
- `standard`: The token standard to filter by
quiet-node marked this conversation as resolved.
Show resolved Hide resolved

There are three possible cases:
1. Only `contractAddress` is provided
quiet-node marked this conversation as resolved.
Show resolved Hide resolved
2. Only `address` is provided
3. Both `address` and `contractAddress` are provided

1. Validate the block range and take the timestamp range
2. Get the logs from the MN for the given timestamp range and filter them by the address and contract address via the topic parameters
3. Filter the results by checking the contract address returned in the ERC registry and if it matcches the standard
4. Transform the logs to the format of the `TokenTransferEvent` interface


#### Error Handling


### Performance Considerations

1. Possibly restrict the number of logs returned by the MN to a maximum of 10000
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We currently have a max on the logs endpoint we can likely match that

Copy link
Collaborator Author

@konstantinabl konstantinabl Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, investigating this. Seems like its 100 so added this as a value @Nana-EC

2. Possibly restrict the block range to a maximum of 10000 blocks

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the restrictions at etherscan side?

they say "This API endpoint returns a maximum of 10000 records only." which is lees, because we are talking a about transactions (records)

I suggest to have the same limits if possible

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite sure what you mean. In etherscan 10000 records limit is for this endpoint, which we are not going to implement. The limit i mentioned here is for this one,
https://docs.blockscout.com/devs/apis/rpc/account#get-token-transfer-events-by-address, where they have limit for 10000 blocks

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you see the limit in code? I am not able to see it on the doc. Only
Up to a maximum of 10,000 token transfer events. Also available through the GraphQL token_transfers query.

no really seeing 10k blocks limit

3. Consider the amount of resources required to check the ERC registry for each log
quiet-node marked this conversation as resolved.
Show resolved Hide resolved

### Security Considerations

1. **Input Validation**
- Validates block ranges
- Validates address
- Validates contract addresses


### Testing Requirements
Ferparishuertas marked this conversation as resolved.
Show resolved Hide resolved

1. **Unit Tests**
- Test log filtering logic
- Test address matching
- Test token standard detection

2. **Acceptance Tests**
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Acceptance tests should be in teh form of E2E features a User would go through with a focus on the input and the outputs

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Edited them : )

- Test Mirror Node interaction
- Test complete flow with real data
- Test error scenarios
- Test different token standards

3. **Performance Tests**
- Test with large block ranges
- Test with high-volume contracts

### Future Improvements


### Dependencies

- ERC registry
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How so, please expand

Copy link
Member

@quiet-node quiet-node Jan 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 on this. Please expand on how ERC Registry is involved.



## `getTokensOwnedByAddress`

### TBD
2 changes: 1 addition & 1 deletion packages/relay/src/lib/eth.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
/* -
/*-
*
* Hedera JSON RPC Relay
*
Expand Down
Loading