Measuring Developer Relations has been a topic of discussion, debate, confusion, and uncertainty for the duration of DevRel’s existence as a discipline. How do we measure Developer Relations?
Over the course of my career leading Developer Relations (from manager to executive roles), I have seen dozens of articles and presentations sharing philosophies for measuring DevRel, or suggestions for hundreds of different metrics to track. However, there are very few resources that tell anyone exactly how to measure Developer Relations in a quantifiable, wholistic sense.
Do you want to know how to measure Developer Relations in meaningful terms that your C-suite will understand? Do you want to prove the business value of Developer Relations at your organization, and do so in dollars? Over the last several years, I’ve steadily been building up resources, research, experiments, plausibility checks, conversations, and stakeholder studies to really figure out: just how the heck do we actually measure Developer Relations?
As a result of years of thinking about this, I came up with a framework: Keystone DevRel Metrics. While I won’t promise that the framework in this article will solve 100% of your problems, I hope that I can get you heading in the right direction — with significant momentum. This isn’t a silver bullet, but it should give you something solid to start with.
SPECIAL THANKS And importantly: I’m standing on the shoulders of greats, here. Over the years, dozens of DevRel experts have contributed amazing resources, insights, research, and experimentation toward figuring out how to measure Developer Relations. I link to articles by some key folks in the Developer Relations community in the article: check out their work!
I also want to extend special thanks to Balazs Kemenesi and Sanjay Sarathy, coworkers from the Developer Experience group at Cloudinary. Balazs and Sanjay were pivotal in validating this framework, crunching very real numbers, and helping me carry this over the finish line. Thank you!
Introduction: How to Measure Developer Relations
Metrics are sometimes seen as an enemy by Developer Relations practitioners, and this can happen because DevRel is classically hard to measure in an organized, straightforward way. You should learn about different types of metrics, and how they’re meaningful in the context of Developer Relations. Kurt Kemple wrote an outstanding Developer Advocate’s Guide to Metrics and Reporting and I highly recommend that you read it before going further in this blog post. In his article, Kurt discusses:
Understanding the different types of metrics and reporting that are relevant to advocacy will help you take advantage of this Keystone DevRel Metrics framework.
Scalable Developer Growth and Engagement
Another crucial topic in measuring Developer Relations is scalability. I created a flywheel describing how to sustainably drive developer growth and engagement at tech product companies. The flywheel tells the story of the developer’s journey, from the perspective of a Developer Relations practitioner sitting within a company. You can read about the Developer Empowerment Flywheel in this blog article.
When it’s time to measure Developer Relations, it’s important to remember that DevRel touches developers many, many times along their continued journey, all the way from “uninitiated” to “power user.” We need to inspire, educate, activate, engage, and empower developer communities in a large variety of different ways — all while acting sustainably. Metrics are affected at every stage of the user journey, and different initiatives hold different value at different times as well. You will see this pattern often in the article.
Key Performance Indicators
Key Performance Indicators (or KPIs) help us understand how we’re growing and making progress. KPIs should be used in conjunction with trusted metrics. Setting KPI goals that are measured by percentage growth month over month helps teams to focus on performing scalable efforts rather than fixating on a single goal number each month or quarter. When goals are set to grow over time, contributors concentrate their strategies, tactics, and overall efforts on initiatives that continue to expand exponentially to continually generate outcomes, rather than starting and concluding with a single outcome.
Some general KPIs that are commonly used to measure Developer Relations efficacy include things like:
- Growth in content and documentation traffic
- Improvements implemented as a result of developer feedback
- Developer experience feedback implemented
- Developer product feedback implemented
- Growth in the number of developers reached through trainings, workshops, presentations, podcasts, videos, etc.
- Developers reached through event sponsorships
- Developers reached through partnerships with third party companies and co-marketing initiatives
- Developer program membership growth
- Developer community contributions
- Developers certified
- Developer campaign engagement
- Developers reached through partnership engagements
- Engagement / click-copies of code samples and snippets
There are several different types of qualitative and quantitative metrics that we can look at. They should be judged on their own merits and are useful in a variety of different ways.
A Note About the Developer Story
It’s also important to take into account that relationships are the foundational operating principle of Developer Relations. Relationships are classically difficult to quantify, but are fundamentally the key that holds human society together. Translating relationships into numbers distills them in a way that can ultimately lack dimension. Therefore, it is important to not only define DevRel metrics by the numbers, but also by the story and impact that they represent.
Upon encountering an interesting product, a developer goes through many touchpoints — and each adds value until we get to a consequential outcome, or series of outcomes. We really like to measure disparate points in the developer journey, but the complete picture is incredibly important. In order to understand how Developer Relations imparts business value, we need to look at the total value across the stages of the developer story. Keep this in mind, because it will come up again later.
Understanding the Business Value of Developer Relations
To many stakeholders, “business value” means opportunities that translate into dollars, or potential dollars. How do we do this in Developer Relations, when our focus is relationships? Measuring relationships in monetary value often feels antithetical, and it doesn’t come naturally to many folks in Developer Relations. So in that case, let’s apply the “Developer Relations touch” to standard company metrics.
Define Your Keystone Metric Value
So where do we start? In order to demonstrate business value (or the financial potential) of Developer Relations, you should first identify a well-understood metric that the company already uses that has intrinsic value associated with it. What does this mean?
In architecture, a keystone is “the wedge-shaped stone at the apex of a masonry arch or typically round-shaped one at the apex of a vault. In both cases it is the final piece placed during construction and locks all the stones into position, allowing the arch or vault to bear weight” (from Wikipedia). A well-defined “keystone” metric is crucially important to the construction of the Keystone DevRel Metrics framework.
What is a Keystone Metric?
In this framework for how to measure Developer Relations, the keystone metric is the metric by which we assign value to most of what Developer Relations does.
In order to measure Developer Relations, first find your keystone metric. Depending on the type of organization, products and services provided, and how your company makes money and retains users, this could be any of the following (or other metrics not mentioned):
- 1 signup / registration RECOMMENDED
- 1 Monthly Active User (MAU)
- 1 average Sales deal
- 1 download / install
- 1 monthly active Open Source contributor
A keystone metric should be something that is already:
- well understood by anyone at the company
- a standard, reliable KPI for non-DevRel teams (e.g., Marketing, Sales, etc.)
- tracked and measurable (the company knows how many of them are generated on a regular basis)
- associated with a dollar value equivalent, or able to be converted to a dollar value through a logical, data-driven formula
In fact, a good feasibility check for a keystone metric might be this:
In the past, have you been asked to measure Developer Relations with a specific business metric? For example: “How many registrations does DevRel drive each quarter?”
Have you then struggled to answer because DevRel does measure this, but alone, it doesn’t even come close to quantifying everything DevRel does? If the answer is yes, it might be a really good keystone metric to use with this framework.
I strongly recommend choosing a keystone metric that does, in fact, correlate to something that Developer Relations can track in complete units. Of course, this metric is only part of the story (hence why you’re reading this article), but having something traceable greatly simplifies the value math later. This is why I like to use registrations / account signups.
Keystone Metric Value
The key to measuring Developer Relations with keystone metrics is assigning value. A typical metric by which other departments swear often already has an equivalent financial worth that is understood at a company. If it doesn’t, then proposing a monetary value will likely be met with resounding support, agreement, and collaboration from other departments.
What we ultimately want to get to is not simply the number of actual units of the keystone metric. We want to measure Developer Relations in value to the business — which is potential revenue.
For example: when I led Developer Relations at Cloudinary, our keystone metric was registrations. A registration at Cloudinary was one free account sign-up. A registration had a monetary value associated with it, so each successful registration was worth $X US dollars.
The value of a keystone metric can be determined through a formula derived from a combination of common sense, understanding of revenue, conversion rates, and other data readily available to the company through Sales, Marketing, customer retention data, and other tracking avenues. Either your company already knows the value of the metric you chose, or you can work cross-functionally to establish it. Your final outcome should be a dollar value that represents the worth of one unit of whatever keystone metric you choose.
Keep your keystone metric value in mind as we move forward.
DevRel Metrics Categories
The Keystone DevRel Metrics framework uses four main categories of metrics that are impactful indicators of success at different stages of the developer user’s product journey:
As I mentioned earlier, there are so many KPIs that we could measure, it can become a mess of numbers, and sometimes we don’t even know why we’re measuring certain things, or what they mean. By categorizing metrics into a few impactful groups, we can assign value to them. We can also explain their meaning much more easily to stakeholders at a company if we use a manageable number of categories — as opposed to overwhelming them with a massive dashboard showing every single thing we’re capable of measuring.
So what do the four categories represent, and how do we measure Developer Relations using them?
Reach is a volume-focused metric. The primary goal for Reach is to drive brand recognition and product visibility at scale across developer communities. For example, Reach is important if you want to:
- become known to developers who have never heard of you
- reinforce and increase familiarity with developers
- generate curiosity and interest in the product to increase awareness
Brand visibility is related to how many people encounter the brand. Examples of this could be placement of the company logo on conference signage, logo presence on talk recording videos, or blurbs or sponsorship ads on content. Brand recognition is the ability of audiences to correctly identify a particular product or service through visual or audio cues (i.e., seeing the logo or hearing the name).
- Brand visibility $0.50
- Logo placement or inclusion on signage, digital assets, etc.
- Event attendees who can potentially see event sponsorship booth branding
- Potential views / listens for sponsored product blurbs in newsletters, tweets, videos, podcasts, etc.
- Brand recognition (non-activation / no product interaction) $1
- Direct visitors to your sponsorship booth
- Interest in company swag
- Leads lists from event registrants who opted into being contacted by sponsors
- Badge scans (i.e., with no further activations or meaningful conversations)
Value of Reach
The value to the business of Reach in financial terms is lower than other categories due to its high volume and low traceability. While most of the subsequent categories can be measured in percentages of the keystone metric, reach is important but less attributable, meaning it should be valued differently. In the past, I have valued it at $0.50 – $1 per unit. Take into account that your numbers for Reach will likely be in the thousands or even tens of thousands.
The value assigned here should make logical sense with regards to the overall ways that your company tracks the keystone metric value. Derive your value of Reach by the value of your keystone metric, company revenue, how much you need to expose new developer audiences to your brand, and other important factors. For example, perhaps you value Reach as 1/100th of a registration: that is, one out of one hundred people register after encountering the brand during any single Reach opportunity.
The primary goal for Awareness is to share information about the product and its capabilities with as many relevant developer audiences as possible with the longer-term goal to increase brand penetration and continued Engagement.
Awareness refers to knowledge about and well-informed interest in a particular product. Measure Awareness with KPIs associated with consumption of educational content, conversations indicating interest in the product, and other similar activities.
- Content awareness
- Net new video views (internal and sponsored)
- Net new blog / article pageviews
- Net new podcast downloads / listens
- Net new livestream views
- Event awareness
- Conference talk attendance (does not include Workshops or Meetups, as these are higher-touch and therefore categorized differently)
- Picked up product information handouts: shows active interest in product rather than just interest in swag
- Meaningful conversations (in person or online) with developers who want to learn about the product
Value of Awareness
The value to the business of Awareness in financial terms is measured in terms of the keystone metric.
For example, if you’re using registrations as your keystone, perhaps an average of one in ten content consumptions leads to an account creation. Therefore, 10% of Awareness metrics can potentially be considered net new registrations, meaning a value of .1 × $REGISTRATION 10% OF 1 REGISTRATION per unit.
Again, you should calculate the percentage using logic regarding your specific keystone metric. I personally like to use registrations, when possible. However, a registration is easier to acquire than a Monthly Active User, so make sure to adjust your percentage accordingly. And some companies don’t index on registrations, but instead focus on things like open source contributors, downloads, etc.
The primary goal for Engagement is to connect with developers to increase brand penetration and activate audiences to continue to engage with the company and product over time.
Engagement can happen whether or not developer audiences have previous Reach or Awareness touchpoints. I recommend that your keystone metric correlate with one engagement unit. This makes measuring Developer Relations easier than translating the keystone metric through percentages or guesswork across all categories.
- Event engagement
- In-depth conversations / product questions that can potentially convert to DevRel Qualified Leads
- Activations (e.g., watched a product demo, participated in a hands-on product challenge)
- Workshops, hackathons, meetups
- High value due to depth of content and commitment
- These may convert to DevRel Qualified Leads if you gather their information for later contact
- Tracked registrations
- Active webinar / livestream / office hours engagement in Q&A
- Subscribers (commitment to continued content delivery)
- YouTube subscribers
- Newsletter subscribers
- Downloads & usage of Developer Relations / Developer Experience built libraries 10-25% OF REGISTRATION
- E.g., OSS SDKs, npm packages
- Example: if your keystone metric is registrations, you might consider that 10% – 25% of downloads could be considered net-new registrations, on average (taking into account upgrades, existing users starting new projects, etc.)
Value of Engagement
Because I like to use registrations as the keystone metric, value in the engagement category is very easy to calculate. Using registrations, the value to the business in financial terms is one registration ($REGISTRATION 1 REGISTRATION) per unit (or percentage of unit as denoted above).
If you are using a keystone metric that does not translate into one unit of engagement, make sure that you use other metrics as well as robust plausibility logic to define your engagement formula.
DevRel Qualified Leads
DevRel Qualified Leads are an impactful standard for DevRel metrics. This metric was introduced by Mary Thengvall in her excellent article DevRel Qualified Leads: Repurposing a Common Business Metric to Prove Value. DRQLs lead to continued partnership and are each worth dozens to potentially hundreds of individual registrations.
DevRel Qualified Leads KPIs
DevRel Qualified Leads are straightforward to quantify. Each quarter, you will only have a few DRQLs, but each is highly impactful to sustainable, scalable Developer Relations and to the business. Examples of DRQLs are:
- Leads for co-marketing and/or content collaboration
- Leads for developer program membership (e.g., community ambassadors, technical guest authors, etc.)
- Leads for Product-based programs (e.g., Developer Advisory Board, early access groups, beta testing groups, etc.)
- Leads for Business Development, such as partner integrations
- Leads for Sales / leads for connections to decision-makers with purchase power
- Community developer ambassadors
- Membership growth
- Active program participation activities / contributions / content creation
- Top champions / superusers
- DevRel partnerships with companies and/or individuals in the community
Data Required for DevRel Qualified Leads
The following is the data you should have in order for something to be considered a DevRel Qualified Lead. You may have meaningful conversations that fall under other metrics, but in order to be an official DRQL (i.e., in order to qualify the lead), the following information is necessary:
- Personal information REQUIRED
- Email / contact
- Goals / intent / potential impact REQUIRED
- Information entered in CRM database OPTIONAL
- Followup action item, continued relationship, or handoff REQUIRED
Reporting and Tracking DevRel Qualified Leads
The Developer Relations team should report new DevRel Qualified Leads using a form (accessible to the DevRel team) that intakes all of the information indicated above.
DevRel Qualified Leads can then be tracked and followed up on in a database (e.g., Airtable, Monday, Excel, etc.). Make sure to include the following when tracking DRQLs:
- Status (e.g., Planning, In Progress, Done, Handed Off, Blocked, etc.)
- Segmentation (see below)
- Initial contact (name of the Developer Relations team member who established a connection and submitted the DRQL)
- Where did the DRQL originate? (e.g., Discord, name of event, etc.)
Do not forget to update the status of the DevRel Qualified Lead, or transfer the information to a more appropriate format if it leads to an ongoing relationship or collaboration.
Segmentation of DRQLs
DevRel Qualified Leads should be segmented based on where they should go next. Some should stay within Developer Relations. Others might be sent to Marketing, Product, Sales, Business Development, HR, etc. Here are some examples of segmentation of DRQLs:
- Leads for co-marketing and/or content collaboration
- DevRel content collaboration (blog, video, presentation, podcast guest, etc.)
- Marketing handoff
- Guest blog post authorship
- Leads for developer program membership
- Community developer ambassadors
- Developer Advisory Board
- Developer guest author program membership
- Forum / Discord community moderator
- Leads for Sales
- Leads for connections to decision-makers with purchase power
- Leads for Business Development / partner integrations
- Technology partner handoff
- Integration development interest
- Leads for Human Resources / recruiting / hiring
Follow Up is Required
In order to qualify as a DevRel Qualified Lead, some form of followup is mandatory. A conversation or connection is not a DevRel Qualified Lead unless further actions can and should be taken. Actions may include warm handoffs to other departments. Actions could potentially be additional sponsorships, product feedback submitted, technology partnerships, leads applying to join programs, content partnerships, recruitment for employment, etc.
Value of DevRel Qualified Leads
Though you will have vastly fewer DRQLs than total sums in the other categories, one single DRQL could potentially:
- drive tens, hundreds, or even thousands of registrations
- lead to an enterprise sales deal
- lead to an official partner integration
- drive awareness for hundreds or thousands of developers
- …and much more
Therefore, each individual DevRel Qualified Lead’s average value to the business in financial potential should be calculated accordingly. Again, I like to use registrations. In previous framework iterations, I worked with the team to determine that one DevRel Qualified Lead is (on average) worth 25 REGISTRATIONS (25 × $REGISTRATION). This may seem like a conservative estimate, but when taking all of the other categories into account along with plausibility checks, it’s the value that worked best for that particular company’s DevRel.
Your mileage may vary, so make sure that you do the following…
Measure Developer Relations… Then Measure Again
If you’ve ever physically built something, you may have heard the construction idiom “Measure twice, cut once.” This saying means you should always double-check your work to prevent mistakes that are difficult to reverse later. Thankfully, this is not necessarily the case when it comes to measuring Developer Relations, and we can always go back later to adjust the way we do things if something is out of place.
However, there are benefits to getting it mostly right early on, even if it requires a little more work upfront. This is particularly true if one of your goals for measuring Developer Relations is to justify the business value of Developer Relations to key company stakeholders or C-suite executives. And the way to find out if you’ve gotten it mostly right is to — you guessed it — “measure twice.”
Run the Numbers
The Keystone DevRel Metrics framework assumes that your DevRel team already tracks many different KPIs, which should now be categorized appropriately. Now use your keystone equations to calculate how much value Developer Relations has generated in the last three months (the past quarter), and since the beginning of the year, if possible.
Once you’ve done that, you have some baseline valuations to work with. Next, gather fiscal metrics for the company overall. If you need to, you can simply start with total revenue for the same time periods. If you have more access to metrics, determine the cumulative value of the keystone metric from across the company (i.e., if you’re using registrations, what’s the total value of all registrations the company generated?). Remember, your keystone metric should be something the company tracks and values in general, so this information should be readily available to you.
Once you have the groundwork laid to measure the value of Developer Relations, compare your numbers to the overall company numbers. This is a plausibility check. When the audience is key company leadership, a Board of Directors, or other important stakeholders, you must verify plausibility before sharing how much value Developer Relations brings to the business.
Look carefully at your valuations, and then ask and answer these questions:
1. Do your calculation formulas make logical sense?
Maybe you decided that workshop participation was worth five registrations. Is that really plausible if workshop attendees only sign up for one account in order to participate? Maybe you said every talk attendee is worth 50% of a registration. Is it logical to assume that an average of one out of every two people who listened to a talk then signs up? If not, you should revisit your calculation formulas and adjust accordingly using common sense and your knowledge of how users engage with the work done by Developer Relations and other teams at your organization.
2. Do your numbers align with company revenue?
If your calculations say that Developer Relations accounts for only 1% of the company’s annual revenue, ask yourself if that’s really true — does it seem accurate? Conversely, if your calculations account for 50% of the company’s annual revenue, does that feel correct? Does it make sense in comparison to the efforts from other departments, like Marketing? If not, reconsider the distribution of value across and within the metrics categories.
3. Do your numbers align with the DevRel budget?
If your calculations say that the value of Developer Relations represents only 20% of the total budget awarded to DevRel each year, does that make logical sense? Will it justify the money the company spends to fund DevRel initiatives? If not, a reckoning could be coming — perhaps the DevRel budget should be revisited, or perhaps something is off in your calculations. Either way, be sure to take a closer look.
Make Adjustments to Ensure Plausibility
If anything doesn’t look or feel right, make adjustments and run the numbers again. To verify plausibility of your implementation of the Keystone DevRel Metrics framework, all three of the above questions should have satisfactory and logical answers supported by the metrics and the mission and directives of the Developer Relations team. In other words: measure twice before you present the value of Developer Relations to the wider company.
Adjust Over Time
Once you’re comfortable with your Keystone DevRel Metrics implementation, don’t stay too comfortable for too long. Metrics are meant to evolve with the growth and fluctuations of a business. Repeat your plausibility checks regularly — every year, at the least. If things need to change, then change them: you should make sure you’re continually adapting to the economy and the financial state of the company. Sometimes that may mean reducing cash burn, or significantly changing your strategic approach to Developer Relations.
This kind of change is normal. Be willing to embrace it, and make sure that you keep your metrics up to date so that they always provide a relevant, contextual representation of the business value of Developer Relations.
Consider the Whole Developer Story
The Keystone DevRel Metrics framework relies on tracking value in four chapters of the developer story. This means the same user should pass through the framework several times along their product journey. One single user’s activity might end up being worth anywhere from less than a dollar to dozens of registrations over time. This is good! This is a realistic representation of value in Developer Relations.
For example, consider these different paths a developer could potentially take:
The Developer Champion
- Developer goes to a tech conference and attends a talk presented by a Developer Advocate on your DevRel team.
- Developer watches a company-sponsored YouTube video from a content influencer.
- Developer registers for an account.
- Developer subscribes to your newsletter.
- Developer joins your ambassador program.
- Developer writes a blog post teaching others how to use your product.
Road to Sales
- Developer goes to a tech conference and visits your booth where they chat with DevRel team members about your product, then interact with a product demo.
- Developer provides their contact information to learn more about the product and connect with Sales.
- Developer registers for an account to create a Proof of Concept for their company.
- DevRel passes the contact information to Sales, and an Account Development Representative reaches out.
- A few months later, an enterprise deal is closed-won with the company the developer works at.
- Developer listens to a podcast where a sponsored pre-roll plays for your product.
- Developer searches for a topic related to your product space, recognizes your product name from hearing it on the podcast, and then visits your website and reads a technical blog post published by DevRel.
- Developer then watches a product tutorial video on your YouTube channel.
- Developer registers for an account.
- Developer downloads, installs, and implements an SDK developed by DevRel to use in their application.
In each of these example stories, the developer comes in contact with Developer Relations initiatives several times, and generates value in each encounter. While granular, multi-touch attribution is challenging to track in many types of Developer Relations activities, understanding the developer story lends depth and context to the value of DevRel. Make sure you keep this in mind when communicating how you measure Developer Relations.
How to Communicate the Business Value of Developer Relations with Stakeholders
While your Developer Relations team likely tracks dozens of different metrics and KPIs (each of which contributes to one of the four keystone metrics categories), the CEO or the VP of Product or the Board of Directors don’t want to see a massive dashboard they’ll then need to translate into value or growth.
For the Developer Relations team internally, the complete, detailed breakdown and distribution is good for:
- forming insights on where to focus, grow, or scale back
- driving and measuring specific DevRel strategies, projects, and campaigns
- refining and understanding the metrics that drive the Keystone DevRel Metrics
Stakeholders, executives, and Boards of Directors are responsible for knowing how the entire organization is performing. They don’t have the bandwidth to go through an entire dashboard or spreadsheet of complex metrics. Share the meaningful, impactful value numbers with stakeholders — the metrics that paint the big picture.
Focus on these things:
- Define the metrics categories: what do Reach, Awareness, Engagement, and DevRel Qualified Leads represent? Clearly explain the categories the first time you present your keystone metrics, then keep these definitions handy for stakeholders to reference later if they need to refresh.
- Value: what are the total calculated values for these categories? (Monthly, quarterly, annually) Also, what is the total value of all four categories combined?
- Growth: month-over-month or year-over-year, how much is the value of Developer Relations increasing? Growth rates should be calculated as percentages: X% more value was generated this quarter compared to last quarter. Show growth rates for each category, and also for Developer Relations as a whole.
If stakeholders are then interested in knowing exactly how you calculated the value of each category, then you can share the KPIs that roll up into each category, your formulas, and the math behind your calculations.
Conclusion: Keystone DevRel Metrics Framework
I know this is a lot of information to take in and process. As mentioned in the beginning of this article, the road to an understandable, straightforward, value-based metrics framework has been long — to say the least. I have been working on the challenge of how to measure Developer Relations and then communicate its value to stakeholders for over five years, across several different types of companies of varying sizes, stages, and product types.
Again, I want to sincerely thank all of the folks who have done — and generously shared — amazing work in this space. This could never have happened without extensive research, experimentation, and learnings from the collective experience of many, many Developer Relations practitioners and leaders.
I know this framework won’t answer everyone’s questions about how to measure Developer Relations, but I do hope it gives you something solid to start from. I look forward to hearing how the Keystone DevRel Metrics framework works for you: what keystone metric you choose, and what adjustments you make in order to fit the needs of your team and organization.