Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: added polling logic to ensure the retrieval of fully mature records from MN #3368

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
169 changes: 114 additions & 55 deletions packages/relay/src/lib/clients/mirrorNodeClient.ts
Original file line number Diff line number Diff line change
Expand Up @@ -754,45 +754,75 @@
}

/**
* In some very rare cases the /contracts/results api is called before all the data is saved in
* the mirror node DB and `transaction_index` or `block_number` is returned as `undefined` or `block_hash` as `0x`.
* A single re-fetch is sufficient to resolve this problem.
* Retrieves contract results with a retry mechanism to handle immature records.
* When querying the /contracts/results api, there are cases where the records are "immature" - meaning
* some fields are not yet properly populated in the mirror node DB at the time of the request.
*
* An immature record can be characterized by:
* - `transaction_index` being null/undefined
* - `block_number` being null/undefined
* - `block_hash` being '0x' (empty hex)
*
* This method implements a retry mechanism to handle immature records by polling until either:
* - The record matures (all fields are properly populated)
* - The maximum retry count is reached
*
* @param {string} methodName - The name of the method used to fetch contract results.
* @param {any[]} args - The arguments to be passed to the specified method for fetching contract results.
* @param {RequestDetails} requestDetails - Details used for logging and tracking the request.
* @returns {Promise<any>} - A promise resolving to the fetched contract result, either on the first attempt or after a retry.
* @returns {Promise<any>} - A promise resolving to the fetched contract result, either mature or the last fetched result after retries.
*/
public async getContractResultWithRetry(
methodName: string,
args: any[],
requestDetails: RequestDetails,
): Promise<any> {
const shortDelay = 500;
const contractResult = await this[methodName](...args);

if (contractResult) {
const contractObjects = Array.isArray(contractResult) ? contractResult : [contractResult];
for (const contractObject of contractObjects) {
if (
contractObject &&
(contractObject.transaction_index == null ||
contractObject.block_number == null ||
contractObject.block_hash == EthImpl.emptyHex)
) {
if (this.logger.isLevelEnabled('debug')) {
this.logger.debug(
`${requestDetails.formattedRequestId} Contract result contains undefined transaction_index, block_number, or block_hash is an empty hex (0x): transaction_hash:${contractObject.hash}, transaction_index:${contractObject.transaction_index}, block_number=${contractObject.block_number}, block_hash=${contractObject.block_hash}. Retrying after a delay of ${shortDelay} ms `,
);
const mirrorNodeRetryDelay = this.getMirrorNodeRetryDelay();
const mirrorNodeRequestRetryCount = this.getMirrorNodeRequestRetryCount();

let contractResult = await this[methodName](...args);

for (let i = 0; i < mirrorNodeRequestRetryCount; i++) {
if (contractResult) {
const contractObjects = Array.isArray(contractResult) ? contractResult : [contractResult];

let foundImmatureRecord = false;

for (const contractObject of contractObjects) {
if (
contractObject &&
(contractObject.transaction_index == null ||
contractObject.block_number == null ||
contractObject.block_hash == EthImpl.emptyHex)
) {
// Found immature record, log the info, set flag and exit record traversal
if (this.logger.isLevelEnabled('debug')) {
this.logger.debug(

Check warning on line 800 in packages/relay/src/lib/clients/mirrorNodeClient.ts

View check run for this annotation

Codecov / codecov/patch

packages/relay/src/lib/clients/mirrorNodeClient.ts#L800

Added line #L800 was not covered by tests
`${
requestDetails.formattedRequestId
} Contract result contains nullable transaction_index or block_number, or block_hash is an empty hex (0x): contract_result=${JSON.stringify(
contractObject,
)}. Retrying after a delay of ${mirrorNodeRetryDelay} ms `,
);
}

foundImmatureRecord = true;
break;
}

// Backoff before repeating request
await new Promise((r) => setTimeout(r, shortDelay));
return await this[methodName](...args);
}

// if foundImmatureRecord is still false after record traversal, it means no immature record was found. Simply return contractResult to stop the polling process
if (!foundImmatureRecord) return contractResult;

// if immature record found, wait and retry and update contractResult
await new Promise((r) => setTimeout(r, mirrorNodeRetryDelay));
contractResult = await this[methodName](...args);
} else {
break;
}
}

// Return final result after all retry attempts, regardless of record maturity
return contractResult;
}

Expand Down Expand Up @@ -895,24 +925,36 @@
}

/**
* In some very rare cases the /contracts/results/logs api is called before all the data is saved in
* the mirror node DB and `transaction_index`, `block_number`, `index` is returned as `undefined`, or block_hash is an empty hex (0x).
* A single re-fetch is sufficient to resolve this problem.
* Retrieves contract results log with a retry mechanism to handle immature records.
* When querying the /contracts/results/logs api, there are cases where the records are "immature" - meaning
* some fields are not yet properly populated in the mirror node DB at the time of the request.
*
* An immature record can be characterized by:
* - `transaction_index` being null/undefined
* - `log index` being null/undefined
* - `block_number` being null/undefined
* - `block_hash` being '0x' (empty hex)
*
* This method implements a retry mechanism to handle immature records by polling until either:
* - The record matures (all fields are properly populated)
* - The maximum retry count is reached
*
* @param {RequestDetails} requestDetails - Details used for logging and tracking the request.
* @param {IContractLogsResultsParams} [contractLogsResultsParams] - Parameters for querying contract logs results.
* @param {ILimitOrderParams} [limitOrderParams] - Parameters for limit and order when fetching the logs.
* @returns {Promise<any[]>} - A promise resolving to the paginated contract logs results.
* @returns {Promise<any[]>} - A promise resolving to the paginated contract logs results, either mature or the last fetched result after retries.
*/
public async getContractResultsLogsWithRetry(
requestDetails: RequestDetails,
contractLogsResultsParams?: IContractLogsResultsParams,
limitOrderParams?: ILimitOrderParams,
): Promise<any[]> {
const shortDelay = 500;
const mirrorNodeRetryDelay = this.getMirrorNodeRetryDelay();
const mirrorNodeRequestRetryCount = this.getMirrorNodeRequestRetryCount();

const queryParams = this.prepareLogsParams(contractLogsResultsParams, limitOrderParams);

const logResults = await this.getPaginatedResults(
let logResults = await this.getPaginatedResults(
`${MirrorNodeClient.GET_CONTRACT_RESULT_LOGS_ENDPOINT}${queryParams}`,
MirrorNodeClient.GET_CONTRACT_RESULT_LOGS_ENDPOINT,
MirrorNodeClient.CONTRACT_RESULT_LOGS_PROPERTY,
Expand All @@ -922,33 +964,50 @@
MirrorNodeClient.mirrorNodeContractResultsLogsPageMax,
);

if (logResults) {
for (const log of logResults) {
if (
log &&
(log.transaction_index == null ||
log.block_number == null ||
log.index == null ||
log.block_hash === EthImpl.emptyHex)
) {
if (this.logger.isLevelEnabled('debug')) {
this.logger.debug(
`${requestDetails.formattedRequestId} Contract result log contains undefined transaction_index, block_number, index, or block_hash is an empty hex (0x): transaction_hash:${log.transaction_hash}, transaction_index:${log.transaction_index}, block_number=${log.block_number}, log_index=${log.index}, block_hash=${log.block_hash}. Retrying after a delay of ${shortDelay} ms.`,
);
for (let i = 0; i < mirrorNodeRequestRetryCount; i++) {
if (logResults) {
let foundImmatureRecord = false;

for (const log of logResults) {
if (
log &&
(log.transaction_index == null ||
log.block_number == null ||
log.index == null ||
log.block_hash === EthImpl.emptyHex)
) {
// Found immature record, log the info, set flag and exit record traversal
if (this.logger.isLevelEnabled('debug')) {
this.logger.debug(

Check warning on line 981 in packages/relay/src/lib/clients/mirrorNodeClient.ts

View check run for this annotation

Codecov / codecov/patch

packages/relay/src/lib/clients/mirrorNodeClient.ts#L981

Added line #L981 was not covered by tests
`${
requestDetails.formattedRequestId
} Contract result log contains undefined transaction_index, block_number, index, or block_hash is an empty hex (0x): log=${JSON.stringify(
log,
)}. Retrying after a delay of ${mirrorNodeRetryDelay} ms.`,
);
}

foundImmatureRecord = true;
break;
}

// Backoff before repeating request
await new Promise((r) => setTimeout(r, shortDelay));
return await this.getPaginatedResults(
`${MirrorNodeClient.GET_CONTRACT_RESULT_LOGS_ENDPOINT}${queryParams}`,
MirrorNodeClient.GET_CONTRACT_RESULT_LOGS_ENDPOINT,
MirrorNodeClient.CONTRACT_RESULT_LOGS_PROPERTY,
requestDetails,
[],
1,
MirrorNodeClient.mirrorNodeContractResultsLogsPageMax,
);
}

// if foundImmatureRecord is still false after record traversal, it means no immature record was found. Simply return logResults to stop the polling process
if (!foundImmatureRecord) return logResults;

// if immature record found, wait and retry and update logResults
await new Promise((r) => setTimeout(r, mirrorNodeRetryDelay));
logResults = await this.getPaginatedResults(
`${MirrorNodeClient.GET_CONTRACT_RESULT_LOGS_ENDPOINT}${queryParams}`,
MirrorNodeClient.GET_CONTRACT_RESULT_LOGS_ENDPOINT,
MirrorNodeClient.CONTRACT_RESULT_LOGS_PROPERTY,
requestDetails,
[],
1,
MirrorNodeClient.mirrorNodeContractResultsLogsPageMax,
);
} else {
break;

Check warning on line 1010 in packages/relay/src/lib/clients/mirrorNodeClient.ts

View check run for this annotation

Codecov / codecov/patch

packages/relay/src/lib/clients/mirrorNodeClient.ts#L1010

Added line #L1010 was not covered by tests
}
}

Expand Down
44 changes: 37 additions & 7 deletions packages/relay/src/lib/eth.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1937,6 +1937,8 @@

if (!contractResults[0]) return null;

this.handleImmatureContractResultRecord(contractResults[0], requestDetails);

const resolvedToAddress = await this.resolveEvmAddress(contractResults[0].to, requestDetails);
const resolvedFromAddress = await this.resolveEvmAddress(contractResults[0].from, requestDetails, [
constants.TYPE_ACCOUNT,
Expand Down Expand Up @@ -2231,11 +2233,7 @@
return this.createTransactionFromLog(syntheticLogs[0]);
}

if (!contractResult.block_number || (!contractResult.transaction_index && contractResult.transaction_index !== 0)) {
this.logger.warn(
`${requestIdPrefix} getTransactionByHash(hash=${hash}) mirror-node returned status 200 with missing properties in contract_results - block_number==${contractResult.block_number} and transaction_index==${contractResult.transaction_index}`,
);
}
this.handleImmatureContractResultRecord(contractResult, requestDetails);

const fromAddress = await this.resolveEvmAddress(contractResult.from, requestDetails, [constants.TYPE_ACCOUNT]);
const toAddress = await this.resolveEvmAddress(contractResult.to, requestDetails);
Expand Down Expand Up @@ -2329,6 +2327,8 @@
);
return receipt;
} else {
this.handleImmatureContractResultRecord(receiptResponse, requestDetails);

const effectiveGas = await this.getCurrentGasPriceForBlock(receiptResponse.block_hash, requestDetails);
// support stricter go-eth client which requires the transaction hash property on logs
const logs = receiptResponse.logs.map((log) => {
Expand All @@ -2341,7 +2341,7 @@
removed: false,
topics: log.topics,
transactionHash: toHash32(receiptResponse.hash),
transactionIndex: nullableNumberTo0x(receiptResponse.transaction_index),
transactionIndex: numberTo0x(receiptResponse.transaction_index),
});
});

Expand All @@ -2357,7 +2357,7 @@
logs: logs,
logsBloom: receiptResponse.bloom === EthImpl.emptyHex ? EthImpl.emptyBloom : receiptResponse.bloom,
transactionHash: toHash32(receiptResponse.hash),
transactionIndex: nullableNumberTo0x(receiptResponse.transaction_index),
transactionIndex: numberTo0x(receiptResponse.transaction_index),
effectiveGasPrice: effectiveGas,
root: receiptResponse.root || constants.DEFAULT_ROOT_HASH,
status: receiptResponse.status,
Expand Down Expand Up @@ -2570,6 +2570,8 @@
// prepare transactionArray
let transactionArray: any[] = [];
for (const contractResult of contractResults) {
this.handleImmatureContractResultRecord(contractResult, requestDetails);

// there are several hedera-specific validations that occur right before entering the evm
// if a transaction has reverted there, we should not include that tx in the block response
if (Utils.isRevertedDueToHederaSpecificValidation(contractResult)) {
Expand Down Expand Up @@ -2839,4 +2841,32 @@
const exchangeRateInCents = currentNetworkExchangeRate.cent_equivalent / currentNetworkExchangeRate.hbar_equivalent;
return exchangeRateInCents;
}

/**
* Checks if a contract result record is immature by validating required fields.
* An immature record can be characterized by:
* - `transaction_index` being null/undefined
* - `block_number` being null/undefined
* - `block_hash` being '0x' (empty hex)
*
* @param {any} record - The contract result record to validate
* @param {RequestDetails} requestDetails - Details used for logging and tracking the request
* @throws {Error} If the record is missing required fields
*/
private handleImmatureContractResultRecord(record: any, requestDetails: RequestDetails) {
if (record.transaction_index == null || record.block_number == null || record.block_hash === EthImpl.emptyHex) {
if (this.logger.isLevelEnabled('debug')) {
this.logger.debug(

Check warning on line 2859 in packages/relay/src/lib/eth.ts

View check run for this annotation

Codecov / codecov/patch

packages/relay/src/lib/eth.ts#L2859

Added line #L2859 was not covered by tests
`${
requestDetails.formattedRequestId
} Contract result is missing required fields: block_number, transaction_index, or block_hash is an empty hex (0x). contractResult=${JSON.stringify(
record,
)}`,
);
}
throw predefined.INTERNAL_ERROR(
`The contract result response from the remote Mirror Node server is missing required fields. `,
);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -346,7 +346,12 @@ export class CommonService implements ICommonService {

const logs: Log[] = [];
for (const log of logResults) {
if (log.block_number == null || log.index == null || log.block_hash === EthImpl.emptyHex) {
if (
log.transaction_index == null ||
log.block_number == null ||
log.index == null ||
log.block_hash === EthImpl.emptyHex
) {
if (this.logger.isLevelEnabled('debug')) {
this.logger.debug(
`${
Expand All @@ -371,7 +376,7 @@ export class CommonService implements ICommonService {
removed: false,
topics: log.topics,
transactionHash: toHash32(log.transaction_hash),
transactionIndex: nullableNumberTo0x(log.transaction_index),
transactionIndex: numberTo0x(log.transaction_index),
}),
);
}
Expand Down
Loading
Loading