How Do We Know the Actuator is Working? Part 1 – Commercial Programs

Follow us @CITOrg or @dihrie or this blog for current information on the new Smart City Actuator.

These days, when interviewing ventures for participation in our acceleration programs we typically get asked some variation on that question…what is your success rate? How do we know you will deliver what you promise? Like most reputable programs, CIT has a long track record of successful outcomes across a large portfolio of companies, and we are justifiably proud of that history. We will cite, as do others, some of the statistics on downstream funding and percentage of successful graduates and, along with a few anecdotes of the success stories, that usually answers the question.

But, while there are several listings of many of the accelerator and similar programs, it has been difficult to do direct comparisons of outcomes. For one thing, only a few programs such as TechStars and Y-Combinator publish fairly complete data sets; other data tends to be fragmentary. It is also true that different programs have different objectives, so no single metric is likely to be applicable across the board. Furthermore, how do you compare the outcomes of very early stage programs with those of, for example, A-round institutional investors where the success rates, returns and institutional imperatives are all different?

So, does the accelerator program you are in make a difference? The short answer is yes; the outcomes vary widely. The longer answer gets a bit wonky, so stick with me here. The results, both in this post and the next ones, point to some fairly significant findings as they relate to our national R&D enterprise.

We got the opportunity to study this in more detail during the DHS-funded EMERGE program, initially under a research grant from the USAF Academy. For this research grant part of the experimental design included standing up the EMERGE accelerator as a way to evaluate its effectiveness in technology innovation for the Government. The initial target market was defined as wearables for first responders. After a few months the outcomes looked quite positive, with the status charts showing lots of green indicators and very few yellow or red. Then the sponsors challenged us: those results look good, but can you prove the program is working, relative to all the other innovation activities we fund. Woo; tall order.

The key turned out to be the industry average model presented in the last post:

Would a comparison of individual portfolio performance against this yardstick provide a method for measuring success? [Spoiler alert] Apparently so. Of course any research study needs its caveats, and the big ones here are first the data were generally taken from publicly published data as of 2015, so results may have changed or may not be completely accurate; second, the analysis was at an aggregate portfolio level…a good graduate research project would be to re-do this type of analysis at an individual company level within portfolios; and third, the EMERGE data in particular were stated as being very preliminary data after only a few months of operation, whereas it generally takes 5-7 years for portfolio results to become reliable.

To start, we looked at internal programs. Using the above reference model, how did the early EMERGE data stack up against our own MACH37 cybersecurity accelerator, and against our GAP funds early stage investment program. The results were quite good (full methodology, references and data are documented in the EMERGE 2015 Final Report). Since this original analysis the GAP fund results have continued to improve with additional later stage funding, the MACH37 results have remained steady as the portfolio continues to grow and mature, and the EMERGE results have declined somewhat but are still above the reference model benchmark results.

In fact these results were so good it looked suspicious. Could we explain how these different approaches got to similar outcome levels, or was there some flaw in the methodology? At a top level at least we were able to rationalize. MACH37 is widely known for the strength of its curriculum and the focused efforts of the partners to secure additional funding for the cohort companies. GAP is known for its highly effective due diligence and mentoring of early stage companies, with the resulting risk reduction making these companies more attractive investments for others. And because EMERGE did not make direct investments but was able to attract a lot of interest from early stage ventures, its cost per “shot on goal” was very low. Since each of these mechanisms is inherent in the model, the results at least passed the initial “sniff test”.

Next, to be sure the model could show some variations in results we added two more external accelerators where the data were available, TechStars and Y-Combinator, both well-known, highly successful, excellent programs. Surprisingly, both performed somewhat below the canonical model. One could hypothesize for TechStars that their global federation style of what are essentially franchises could produce enough geographic variability to explain those data. But Y-Combinator in particular was shocking since they are the godfather of the accelerator business and widely recognized as one of the most successful. What the heck?

A bit more research however told the tale. The published statistics for Y-Combinator at the time showed that 29% of their accelerator graduates received external funding, less than half of the industry average in the model. But, more than 7% of their accelerator graduates made it all the way to a Series B funding event, roughly twice the industry average, and with a Series B investment considered to be $40M or more, more than 4X the industry average. So, very high failure rate in the accelerator end of things, but off the charts probability of success for those who did continue on. In part this reflects their model of huge cohorts, more than 100 new companies at a time each receiving $120K initial investments…$12M investment per cohort! The accelerator essentially acts as a highly selective due diligence process, resulting in high quality deal flow for the Y-Combinator investors.

As Yael Hochberg puts it: “Y Combinator is cashing in on the name it made for itself…[t]hey’re talking about raising a multi-billion dollar late stage fund to take advantage of their model that selects great entrepreneurs rather than mold them.” [emphasis added]

This external validation finally convinced us that the methodology was fairly robust and could produce interesting results. The early stage venture program you are in does make a difference, and we continue to be very proud of the fact that CIT programs consistently perform very strongly in terms of outcomes on these types of objective measures.

The original research effort was funded to also look at the performance of Government research and development programs. How do they stack up? That is the topic of Part 2 of this post, along with some policy implications for the national R&D enterprise.

Next (Thursday 3/16): How do we know the Actuator is Working? Part 2 – Government Programs

 

 

A Tale of Four Cities (with apologies to Dickens)

It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the epoch of belief, it was the epoch of incredulity, it was the season of Light, it was the season of Darkness, it was the spring of hope, it was the winter of despair…” Charles Dickens, A Tale of Two Cities

Since the beginning of 2016, it seems like the worst of times. We have seen a correction in the stock market as the Chinese economic bubble has popped, taking the global oil markets with it, and bringing back the all-too-recent memories of the Internet bubble of 2000 and the financial bubble of 2008 (watch out, 2024!). The misery has spread to the Tech sector. The unicorn, unofficial mascot of Silicon Valley, which had gone from being a rare beast in 2014 to a veritable population explosion in 2015, is once again on the verge of extinction.

Yet the economic talking heads tell us this is normal, that the U.S. economy is doing well and is reasonably insulated from both the Chinese economy and the negative oil shock. That corrections are a necessary part of the market, to restore balance after a period of irrational exuberance. So, what the heck is going on with Tech?

In 2015 I was Principal Investigator for a DHS-funded program called EMERGE, working to leverage commercial business accelerators to help commercially-focused innovative companies bring some of their technology to address needs of the DHS community. As part of this program we were fortunate to get an inside view of four different business accelerator programs in four different cities:

• Tech Wildcatters in Dallas, TX
• TechNexus in Chicago, IL
• Wearable World IoT in San Francisco
• And our own MACH37 here at CIT in Virginia, part of the Washington DC region

Here is what I learned. First, tech innovation does not occur in isolation; it is the result of effective regional innovation ecosystems that include customers, entrepreneurs, funding sources, a high concentration of expertise and ideas, and enough of a support infrastructure to help the entrepreneurs through the early pitfalls. Each of the four accelerator programs above has done an outstanding job of helping build and then leverage their local ecosystem as an integral part of what makes each region grow.

Second, Silicon Valley is not identical to the Tech sector. Although news coverage often glosses over this fact, innovation occurs in many places across the country. I will argue below that while Silicon Valley is indeed unique in many ways, generalizations based on that unique set of circumstances can often be wrong. In the current situation, the doom and gloom based on over-priced investments there is less relevant in other parts of the country.

And so, the four cities.

Dallas – Texas has several innovation centers including both Dallas and Austin. There is a diverse industry base, with concentrations in energy, health care/life sciences and tech, significant university presence, and a good concentration of wealth. Tech Wildcatters has successfully provided leadership to the region’s startup community with special programs in both health care and tech, and most recently going to a year-round program from the more typical discrete sessions. Dallas is a vibrant startup location, although it is unclear what effect the collapse of oil prices may have on access to capital in the region.

Chicago – political issues aside, Chicago has the benefit of a high concentration of Fortune 500 Corporate Headquarters, a robust investment sector and strong University presence. TechNexus has done a masterful job first in priming the innovation ecosystem development 7 or 8 years ago, and now tapping into the innovation needs of Corporate strategic partners who are looking to early stage companies as a source of new products and ideas. If the city can recover from its social strife it is certainly positioned to continue as a significant center of tech innovation.

San Francisco – San Francisco/Silicon Valley is the undisputed investment capital of the world for tech. According to Pitchbook in the third quarter of 2015 more than 27% of all the venture capital invested globally came out of Silicon Valley. China has risen rapidly as both a source and target of VC investment, although the collapse of the economy in China seems certain to be a major setback in this area, as the graph seems to indicate starting in Q4 of 2015. New York ranks third on this list, providing just north of 8% of the globally invested capital.

Yet with all that money floating around it appears that some Silicon Valley investors may have had more dollars than sense. If you look at the number of deals and the dollar amounts as compiled by Pitchbook, the dollars invested continued to rise in 2015 even while the number of deals plummeted, leading to a rapid rise in median valuations.

By comparison, valuations in New York during this same time were only 10% of the San Francisco valuations, an enormous disparity. There are some possible alternative explanations for this disparity (bigger opportunities, move towards later stage investments, etc), but both the anecdotal evidence at the time (“too much money chasing too few deals” was a sentiment we heard more than once) and the subsequent down rounds of investment even for some of the high flyers indicates over-valuation on the part of investors was at least one primary cause of the disparity.

A second point. Why on earth would you want to locate and operate a company in the outrageously expensive environs of San Francisco where none of your employees can afford to live? Or Palo Alto, where Palantir is driving out start-ups by snapping up office space at high rents. Well there are certainly some reasons: if you want to hang with the cool kids, California is the place you ought to be. If you need to raise a billion dollars or so, where else would you go? And certainly if you want frothy valuations during the good times, the target destination is clear.

A recent Harvard Business School study (http://www.hbs.edu/faculty/Publication%20Files/09-143.pdf) hinted at one possible evolution of this trend. According to the study:

“Venture capital firms based in locales that are venture capital centers outperform… [as a result of] outsized performance outside of the …firms’ office locations…”

That is, if you are a VC you want to be in one of the centers of VC activity because there is a strong ecosystem of investors…but, the big returns are to be found by investing in other places. Certainly Silicon Valley is not going away as the primary center of activity. Increasingly however, those investors seem to be syndicating with other groups in places such as Dallas, Chicago or…

Washington DC – The region centered around Washington DC is generally considered to include Maryland, Virginia (or at least Northern Virginia), and DC itself. The Federal Government is a large presence, along with some of the specialty areas such as cybersecurity and data analytics it has helped develop. Health care/life sciences is also a major player in the area, and there are multiple world-class universities that support the ecosystem. The region generally ranks in the Top 10 innovation areas of the country, and the area’s capital investments are growing, actually increasing in the 4^th quarter of 2015 even while investments were declining nationally. One reason for this increase is the growth in cybersecurity, with the potential for more than a billion dollars in cybersecurity investments in the region in 2016. The two biggest areas were health care/bio and software (including cyber), and there is an organized, active ecosystem working to promote the growth of these and other industry sectors.

Conclusions – Clearly the stock market is in correction territory, driven initially by economic issues in China and the energy sector. While the tech sector also appears under pressure, the fundamentals here are very different. In the short term, what appears to be a broad retrenchment in the sector is actually mostly a correction of inflated valuations on the West Coast that are not indicative of the sector as a whole. As Rick Gordon, Managing Partner of the MACH37 Cybersecurity Accelerator puts it: “while Silicon Valley has been out on the great unicorn hunt, we have been building an army of cockroaches…small, fast, nimble, designed to survive a nuclear winter, and available at a reasonable price.”

The age of easy money from building the next mobile app may be behind us, but the advent of autonomous vehicles, personalized medicine, data-driven everything and more will ensure that the tech sector will continue to drive the next wave of innovation and economic growth for decades to come. But it is increasingly likely that the actual innovations will be found in places like Dallas, Chicago and the Washington region even if the investment capital still flows from New York and Silicon Valley.