The OSF Playbook: A Journey to Investing in Future Sci-Tech

Creating a Living Due-Diligence Checklist

After setting up the initial model framework based on preliminary information about the synthetic biology company, we needed to gather more information from domain experts (See Sidebar 1) to assess factors in the model and validate the model’s structure (Step 5 in Figure 1). Because the model required numerical values for each factor, it enforced a certain rigor in our diligence process by making us evaluate each input and document why we chose those values. We translated the impact of technological details crucial to success into a common due-diligence format.

In this way, the model became a living — dynamic — due-diligence checklist that we tailored to this specific company and industry. Just as Atul Gawande advocates checklists during complex surgical situations,1 developing a basic checklist for science investments could help investors avoid mistakes of omission and focus on the most complex factors. Our understanding of the company and the synthetic biology domain grew as we developed the model and translated expert information into model inputs.

Because the purpose of this paper is to discuss issues specific to investing in science companies, we focus here on the technical diligence process, and leave it to the reader to seek out existing excellent advice on evaluating other aspects of a company. Note that evaluating certain aspects, such as the quality of the team, may require customization for science-based companies. For example, Matthew Nordan emphasizes2 that for cleantech, a great team must include the core scientist who invented the technology. This might differ in a software company where the innovation is in the business model rather than the technology.

Sidebar 1: Leveraging Technical Expertise

Technical domain experts are critical to better understand the potential risks and rewards for the company and industry. Experts should have a balance of optimism, skepticism, openness to new ideas, and significant full-time industry experience .

We found that previous full-time industry experience is needed to assess the potential of the technology to be profitable and to identify possible pitfalls of manufacturing scale-up. One challenge with cutting-edge technologies is that there is often little-to-no demonstrated industrial validation, so it may be challenging to find experts with industry experience on particular technologies. In these cases, academic expertise alone is not enough; it must be complemented by expertise from the closest industries available (e.g., industries with similar technologies going after different markets, or analogous industries). In addition to domain experts, it is also essential to have technical generalists to bridge understanding between investment specialists and technical specialists.

For every model input, we assessed not only a base estimate but also a range from ‘surprisingly low’ to ‘surprisingly high’. All due diligence for investments involves making estimates, no matter how good the experts involved are. For some factors, we may have high confidence, for others we may have low confidence and much uncertainty (see Sidebar 2 for more).

We used the base case to estimate the expected value of the investment — the expected return multiple (ERM). The range of estimates enabled us to perform a sensitivity analysis identifying which factors have the greatest influence on the return. These critical factors gave us insight into places to put more resources for due diligence or where company plans and operations should be carefully de-risked.

Sidebar 2: Putting a Number on It

Using ranges from “surprisingly low” to “surprisingly high” rather than single number estimates incorporates real-life uncertainty and enables a sensitivity analysis to highlight which factors might be most critical in an investment decision.

Note that surprisingly low and surprisingly high estimates are not necessarily the same as realistic ‘pessimistic case’ and ‘optimistic case’ values. They are values that an expert would consider on the surprisingly low end or the surprisingly high end of possible estimates, and we use them to try to capture the broadest set of scenarios that might happen. Experts can provide their own ranges and the modelers can choose to combine them or to run separate models for each expert (computation is cheap). It is important to document every set of estimates to explain how they were obtained.

Where factors are risk assessments (probabilities of success), they are measured on a range from 0 to 1, where 0 means ‘this cannot work’, 0.5 is a toss-up and 1.0 means that risk is ‘certain to be overcome’. For example, if we are considering the skill set of the team, we may assign a value of 0.7 if the team has a good grasp of technical skills and competent leadership, but industrial experience from a different industry. (E.g. “overcome leadership risks” could be rated: surprisingly low = 0.3, base = 0.7, surprisingly high = 0.9.)

Meanwhile, keep in mind that base estimates are used to calculate the expected value of the investment. Does a specific base estimate need closer scrutiny and deserve more investment of resources in due diligence?

Using our research on the company and its competitors, including materials from the company, industry/analyst reports, and additional resources, we developed an initial list of questions together with the domain experts. The model required answers to these questions (see example questions in Sidebar 3). We then conducted multiple meetings with the company and with our domain experts to gather information, each time generating follow-up questions until we had probed the main identifiable risk factors in enough depth to allow us to enter estimates into the model (Step 6 in Figure 1).

Sidebar 3: Knowing the Right Questions to Ask

Example Questions for Science Companies:

  1. What is the technical concept?  Does it hold up to scrutiny by experts?
  2. Is the scientific basis for the profit model sound?  In the ideal case, does the math work out for this technology to make money considering costs of development and manufacturing?  
  3. What are the main risks? (technical and otherwise) What are the key milestones to de-risk?  What data exists and how strong is it?  
  4. What are the risks associated with successfully scaling up manufacturing from bench/prototype to commercial production? This is a common pitfall for many science-based companies.3 The technological process for making a prototype may be vastly different from scaling up the manufacturing process to make thousands of the same item and may require a step-change in technological development, not just an incremental expansion. There are significant risks if manufacturing cannot be performed rapidly enough at low enough cost.  What experiments or milestones might de-risk scale-up?
  5. What is the intellectual property strategy? What IP has the company secured?  How easy is it for a fast follower to compete?
  6. Who are the main competitors? There may be “technical competitors” with similar technologies who may be pursuing different markets, and/or “market competitors” with different products but pursuing similar markets.
  7. What is the technical advantage over competitors? How could it provide differentiation? What are the moats or barriers against competitors? (technical and otherwise?)
  8. Is the team capable of the level of technical execution required?  Is the original inventor part of the team?

Example Questions for Domains / Industries:

    1. What is the industry clockspeed? (e.g., software is fast, pharmaceuticals is slow) How does industry speed influence timelines to product, revenue, exits?
    2. How capital intensive is the industry? What level of investment is needed? Implications for competition, ability to sustain runway?  Are there any trends to decrease costs of de-risking? (e.g., falling cost of DNA synthesis)
    3. What are examples of typical good investments in this domain and how often do these occur? Typical poor investments?  What is the return-on-investment landscape for the domain, and how does this affect investment strategy?  (e.g., few, unlikely sky-high returns vs. more modest returns with a higher probability of success)
    4. How does the nature of IP in the domain affect strategy and outcomes?  (e.g., compare software to pharmaceuticals)
    5. Are there network effects? How do they influence market dynamics? (e.g., winner-take-all)
    6. What is the rate of commoditization? What is the risk of being leap-frogged in a race to the bottom
    7. What are typical milestones or opportunities for exits?
  1. Atul Gawande, “The Checklist: If something so simple can transform intensive care, what else can it do?”, December 2007, The New Yorker. http://www.newyorker.com/magazine/2007/12/10/the-checklist
  2. Matthew Nordan, “What Makes a Great Cleantech Team?”, September 2012. http://mnordan.com/2012/09/24/what-makes-a-great-cleantech-team/
  3.  “But, I think for manufacturing, very often people think of manufacturing as just some rote process of making copies. Which, actually, it isn’t. Manufacturing is building the machine that makes the machine. If you think the machine is important, well, building the machine that makes the machine is also extremely important, and more often than not, what I’ve found is the manufacturing is harder than the original product. For example, at Tesla we can make one of a car very easily, but to make thousands of a car with high reliability and quality and where the cost is affordable, is extremely hard. I’d say, maybe 10 times harder than just making one prototype – maybe more.” -Elon Musk, Interview at MIT Aero/Astro Centennial Symposium, October 2014. http://webcast.amps.ms.mit.edu/fall2014/AeroAstro/index-Fri-PM.html