Understanding the code that produces the number of built objects in the log

[sadeka4]

I am trying to understand the code that produces the number of built objects in the log. An example looks like this.

03:30:07 46 of 59 OK created sql table model my_dataset.fct_generation

I'd like to know how DBT generates a total of 59 objects in the above line. Which resource types are considered to be included while counting this total and which objects are actually run by DBT afterwards.

Any help would be greatly appreciated!
Thank you.

[QMalcolm]

Hi @sadeka4 so the log in question is an instance of the LogModelResult event. There are additional similar events in dbt-core: LogFreshnessResult, LogNodeNoOpResult, LogSnapshotResult, LogSeedResult, and LogTestResult. Something similar about all of these is that they all have a total attribute. This total is the number of nodes selected for execution in the given run, which is calculated in _runtime_initialize of GraphRunnableTask. We can see in this function that the total num_nodes is the "selected for execution" nodes that are one of

• Exposure
• UnitTest
• SavedQuery
• Source
• "nodes" (Can be any "ManifestNode")
  • Analysis
  • SignularTestNode
  • HookNode
  • ModelNode
  • SqlNode
  • GenericTestNode
  • SnapshotNode
  • UnitTestNode
  • SeedNode

There is one event that is slight different, LogBatchResult. The total number for these log events is the number of batches being executed for the given microbatch model. Its log line will also look slightly different, something like 03:30:07 BATCH 1 of 7 OK created batch...

[sadeka4]

Many thanks, @QMalcolm for clarifying this. Much appreciated!