Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zuora model updates #34

Open
wants to merge 9 commits into
base: master
Choose a base branch
from
Open

Zuora model updates #34

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
8646706
update zuora sql model
dwallacerjm Apr 12, 2016
c4aaaa7
update zuora subscriptions with charges and amendments model sql
dwallacerjm Apr 12, 2016
0a397bf
update zuora sql modesla dn alter dbt_project yml to only build zuora…
dwallacerjm Apr 12, 2016
153be59
more zuora subscription model updates
dwallacerjm Apr 13, 2016
908d1ab
small fix to subscription w charges and amendments model
dwallacerjm Apr 13, 2016
7eda661
add an analysis template for zuora new business metrics
dwallacerjm Apr 13, 2016
c1625f6
reset dbt_project.yml to defaults
dwallacerjm Apr 13, 2016
ca70f42
updates to yevgeniys mrr analyses to make sure they work with new model
dwallacerjm Apr 13, 2016
d7af70b
adjust yevgeniys total_mrr_by_month analysis to accommodate new zuora…
dwallacerjm Apr 13, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions analysis/mrr/active_mrr.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ charges_for_active_plans as
from {{env.schema}}.zuora_subscriptions_w_charges_and_amendments
where
-- make sure the subscription is active
subscr_status = 'Active'
sub_status = 'Active'
and
(
-- make sure the rate plan charge is current
Expand All @@ -14,7 +14,7 @@ charges_for_active_plans as
and
rpc_end >= current_date
)
or subscr_term_type = 'EVERGREEN'
or sub_term_type = 'EVERGREEN'
)
and rpc_last_segment = TRUE
)
Expand Down
18 changes: 9 additions & 9 deletions analysis/mrr/total_mrr_by_month.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,11 @@ charges_up_to_each_date as
(
select
date_day, date_month, account_number, mrr, rpc_start, rpc_end, rpc_last_segment,
amend_start, amend_id, subscr_term_type, subscr_start, subscr_end, subscr_id,
subscr_name, subscr_version,
amend_start, amend_id, sub_term_type, sub_start_date, sub_end_date, sub_id,
sub_name, sub_version,
dateadd(month,1,date_month) as date_month_plus_one,
max(date_day) over (partition by subscr_name, dateadd(month,1,date_month)) as max_subscr_trunc_date,
max(subscr_version) over (partition by subscr_name, date_month) as max_subscr_version_within_date
max(date_day) over (partition by sub_name, dateadd(month,1,date_month)) as max_subscr_trunc_date,
max(sub_version) over (partition by sub_name, date_month) as max_subscr_version_within_date
from dates a
left join subscriptions b
on 1=1
Expand Down Expand Up @@ -58,13 +58,13 @@ all_charges_by_month as
)
or
(
subscr_term_type = 'EVERGREEN'
and subscr_start <= dateadd(month, 1, date_month)
sub_term_type = 'EVERGREEN'
and sub_start_date <= dateadd(month, 1, date_month)
and
(
subscr_end is null
sub_end_date is null
or
subscr_end >= dateadd(month, 1, date_month)
sub_end_date >= dateadd(month, 1, date_month)
)
and
(
Expand All @@ -75,7 +75,7 @@ all_charges_by_month as
)
)
and date_day = max_subscr_trunc_date
and subscr_version = max_subscr_version_within_date
and sub_version = max_subscr_version_within_date
and dateadd(month, 1, date_month) <= current_date
)

Expand Down
16 changes: 16 additions & 0 deletions analysis/zuora/new_business_metrics.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
select
sum(mrr) as gross_new_mrr,
sum(mrr)-lag(sum(mrr),1) over (order by date_trunc('month',contract_effective_date)) as delta_gross_new_mrr,
((sum(mrr)-lag(sum(mrr),1) over (order by date_trunc('month',contract_effective_date)))/lag(sum(mrr),1) over (order by date_trunc('month',contract_effective_date)))*100 as percent_gross_new_mrr_growth,
sum(mrr)*12 as gross_new_arr,
sum(tcv) as gross_new_bookings,
count(1) as gross_new_subscriptions,
count(distinct account_id) as gross_new_accounts,
avg(mrr) as avg_new_subscription_mrr,
avg(tcv) as avg_contract_value,
avg(datediff(months,sub_start_date,sub_end_date)) as avg_new_subscription_length,
date_trunc('month',contract_effective_date) as date
from {{env.schema}}.zuora_subscription
where sub_version = 1
group by date_trunc('month',contract_effective_date)
order by date_trunc('month',contract_effective_date)
7 changes: 4 additions & 3 deletions dbt_project.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ model-defaults:
materialized: false
enabled: true

#models:
# zuora:
# materialized: true
##models:
## zuora:
## enabled: true
## materialized: false
168 changes: 157 additions & 11 deletions models/zuora/zuora_subscription.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,158 @@
with sub_mrr as (
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dwallacerjm Dave, this analysis should be in the analytics/analysis/zuora folder, not analytics/models/zuora. The general idea is to first create base models which map the underlying zuora tables into views that incorporate only what's needed for subsequent transformations and analysis, including all of the casting and any transformation that may be duplicated down the line. In other words: create an abstraction that makes it easier to write and read SQL down the line.

This is exactly what's happening int the original zuora_subscription.sql, which is a base model for the underlying zuora.zuora_subscription table. There is casting into timestamps and the subscription version is calculated in that view so that we don't have to worry about it down the line.

Also, let's keep subscr instead of sub as this is more descriptive to someone going through this analysis in the future.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Little bit confused here @3mei . This SQL is for the creation of the model. It isn't an analysis at all.

select
id as subscr_id,
status as subscr_status,
termtype as subscr_term_type,
accountid as account_id,
contracteffectivedate::timestamp as subscr_start,
subscriptionenddate::timestamp as subscr_end,
name as subscr_name,
"version#392c30e6081c24fb78ddf6d622de4f33"::integer
as subscr_version,
*
from zuora.zuora_subscription
c.id as sub_id,
sum(cast(a.mrr as decimal(18,2))) as mrr,
sum(cast(a.tcv as decimal(18,2))) as tcv
from zuora.zuora_rate_plan_charge a
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dwallacerjm Once a base model is created, we should not reference the original table. This should be {{env.schema}}.zuora_rate_plan_charge, where the {{env.schema}} indicates that we are using a base model.

join zuora.zuora_rate_plan b
on a.rateplanid = b.id
join zuora.zuora_subscription c
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dwallacerjm same for all of these

on b.subscriptionid = c.id
where a.islastsegment = TRUE
group by 1
),

sub_billing_periods AS (
SELECT
listagg(billingperiod,', ') WITHIN GROUP (ORDER BY billingperiod) AS billing_periods,
subscriptionid
FROM (
SELECT
DISTINCT b.subscriptionid,a.billingperiod
FROM zuora.zuora_rate_plan_charge a
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dwallacerjm If you need to reference the same base model several times, just create a CTE at the beginning of the query so that you hit the underlying table once. For example,

with rate_plan_charge as ( select * from {{env.schema}}.zuora_rate_plan_charge )

then you can simply do select from rate_plan_charge

JOIN zuora.zuora_rate_plan b
ON a.rateplanid = b.id
where a.islastsegment = TRUE
order by a.mrr desc
)
GROUP BY 2
),

sub_charge_types AS (
SELECT
listagg(chargetype,', ') WITHIN GROUP (ORDER BY chargetype) AS charge_types,
subscriptionid
FROM (
SELECT
DISTINCT b.subscriptionid,a.chargetype
FROM zuora.zuora_rate_plan_charge a
JOIN zuora.zuora_rate_plan b
ON a.rateplanid = b.id
where a.islastsegment = TRUE
order by a.mrr desc
)
GROUP BY 2
),

sub_charge_models AS (
SELECT
listagg(chargemodel,', ') WITHIN GROUP (ORDER BY chargemodel) AS charge_models,
subscriptionid
FROM (
SELECT
DISTINCT b.subscriptionid,a.chargemodel
FROM zuora.zuora_rate_plan_charge a
JOIN zuora.zuora_rate_plan b
ON a.rateplanid = b.id
where a.islastsegment = TRUE
order by a.mrr desc
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dwallacerjm The very same join is performed in multiple subqueries. We can simplify this by creating a common CTE.

)
GROUP BY 2
),

sub_distinct_rate_plans as (
select
listagg(name,', ') WITHIN GROUP (ORDER BY name DESC) AS distinct_rate_plans,
subscriptionid
from (
SELECT
DISTINCT b.subscriptionid,
b.name
FROM zuora.zuora_rate_plan_charge a
JOIN zuora.zuora_rate_plan b
ON a.rateplanid = b.id
where a.islastsegment = TRUE
)
group by 2
),

sub_distinct_products as (
select
listagg(name,', ') WITHIN GROUP (ORDER BY name DESC) AS distinct_products,
subscriptionid
from (
SELECT
DISTINCT b.subscriptionid,
e.name
FROM zuora.zuora_rate_plan_charge a
JOIN zuora.zuora_rate_plan b
ON a.rateplanid = b.id
join zuora.zuora_product_rate_plan_charge c
on a.productrateplanchargeid = c.id
join zuora.zuora_product_rate_plan d
on c.productrateplanid = d.id
join zuora.zuora_product e
on d.productid = e.id
where a.islastsegment = TRUE
)
group by 2
)

select
a.id as sub_id,
a.name as sub_name,
a.termtype as term_type,
a.status,
a.renewalsetting as renewal_setting,
a.isinvoiceseparate as is_invoice_separate,
a.currenttermperiodtype as current_term_period_type,
coalesce(lag(cast(b.effectivedate as date),1) over (partition by a.name order by cast(a.version#392c30e6081c24fb78ddf6d622de4f33 as integer)),cast(a.subscriptionstartdate as date)) as sub_effective_date,
cast(a.contractacceptancedate as date) as contract_acceptance_date,
cast(a.contracteffectivedate as date) as contract_effective_date,
cast(a.subscriptionstartdate as date) as sub_start_date,
cast(a.subscriptionenddate as date) as sub_end_date,
cast(a.termstartdate as date) as term_start_date,
cast(a.termenddate as date) as term_end_date,
cast(a.cancelleddate as date) as cancelled_date,
cast(a.createddate as timestamp) as created_date,
cast(a.updateddate as timestamp) as updated_date,
cast(a.originalcreateddate as timestamp) as original_created_date,
cast(a.serviceactivationdate as date) as service_activation_date,
cast(a.version#392c30e6081c24fb78ddf6d622de4f33 as integer) as sub_version,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dwallacerjm All of these castings should be done in the base model.

coalesce(c.mrr,0) as mrr,
coalesce(c.tcv,0) as tcv,
d.billing_periods,
f.charge_types,
g.charge_models,
h.distinct_products,
e.distinct_rate_plans,
a.accountid as account_id,
a.invoiceownerid as invoice_owner_id,
a.createdbyid as created_by_id,
a.updatedbyid as updated_by_id,
a.autorenew as auto_renew,
a.renewaltermperiodtype as renewal_term_period_type,
cast(a."renewalterm#d6921dfe0b38e14b10fb11fbb53af737" as integer) as renewal_term,
cast(a."currentterm#43cd92dff45a80fd1ab8e39f16b6af51" as integer) as current_term,
a.initialtermperiodtype as initial_term_period_type,
cast(a."initialterm#58b70e8fe6414e59d8d03a9eb211ab74" as integer) as initial_term,
a.previoussubscriptionid as previous_subscription_id,
a.originalid as original_id,
a.notes,
a.creatoraccountid as creator_account_id,
a.creatorinvoiceownerid as creator_invoice_owner_id
from zuora.zuora_subscription a
left join zuora.zuora_amendment b
on a.id = b.subscriptionid
left join sub_mrr c
on a.id = c.sub_id
left join sub_billing_periods d
on a.id = d.subscriptionid
left join sub_distinct_rate_plans e
on a.id = e.subscriptionid
left join sub_charge_types f
on a.id = f.subscriptionid
left join sub_charge_models g
on a.id = g.subscriptionid
left join sub_distinct_products h
on a.id = h.subscriptionid
18 changes: 9 additions & 9 deletions models/zuora/zuora_subscriptions_w_charges_and_amendments.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,26 +2,26 @@
with subscr_w_amendments as
(
select
account_number, acc.account_id, sub.subscr_id,
subscr_name, subscr_status, subscr_term_type,
subscr_start, subscr_end, subscr_version, amend_id, amend_start
account_number, acc.account_id, sub.sub_id,
sub_name, sub.status, sub.term_type,
sub_start_date, sub_end_date, sub_version, amend_id, amend_start
from {{env.schema}}.zuora_account acc
inner join {{env.schema}}.zuora_subscription sub
on acc.account_id = sub.account_id
-- add ammendments
left outer join {{env.schema}}.zuora_amendment amend
on sub.subscr_id = amend.subscr_id
on sub.sub_id = amend.subscr_id
)

select
account_number, account_id, sub.subscr_id,
subscr_name, subscr_status, subscr_term_type,
subscr_start, subscr_end, subscr_version, amend_id, amend_start,
account_number, sub.account_id, sub.sub_id,
sub.sub_name, sub.status as sub_status, sub.term_type as sub_term_type,
sub.sub_start_date, sub.sub_end_date, sub.sub_version, amend_id, amend_start,
rpc_start, rpc_end, rpc_last_segment,
min(subscr_start) over() as first_subscr,
min(sub.sub_start_date) over() as first_subscr,
"@mrr" as mrr
from subscr_w_amendments sub
inner join {{env.schema}}.zuora_rate_plan rp
on rp.subscr_id = sub.subscr_id
on rp.subscr_id = sub.sub_id
inner join {{env.schema}}.zuora_rate_plan_charge rpc
on rpc.rate_plan_id = rp.rate_plan_id