How Do We Know the Actuator is Working? Part 4 – Synthesis and Policy

Follow us @CITOrg or @dihrie or this blog for current information on the new Smart City Actuator.

Time to pull together the thoughts and data from the previous three posts. Three sections here: how do we know the Actuator is working; are there ways to improve the commercialization success of invention organizations such as Universities and National Labs; and, are there ways to improve the outcomes of the national R&D enterprise

The Actuator

OK, back to the original question..how do we know the Actuator is working? As a participant you will know fairly quickly how it is working for you, once you learn to smooth out the day-to-day highs and lows of being an entrepreneur. However overall at a portfolio level we also carefully track performance metrics against industry norms, and our performance here is very strong. We use these metrics to fine-tune the program content and focus and inform specific mentoring actions; this continued engagement over the longer term is a strong signal that the Actuator continues to work for you. In addition to investment metricReference model fulls, we will also track performance in terms of various paths to market that may or may not involve direct financial investment, such as revenue growth and job creation of the companies we mentor.

But those metrics for a portfolio can take 5-7 years to fully mature. In the intermediate term we track leading indicators of later success. Some of these include pilot opportunities, early customer adoption and similar measures of market traction, even things like press coverage. Where needed, we also use these metrics to indicate additional areas where the Actuator can provide ongoing support to our graduates as they mature their businesses.

For the short term the primary assurances are the combined experience of our CIT and Smart City Works staff in the specific market verticals we are addressing, our extensive direct experience in early stage investing, a deep understanding of accelerators and best practices about what it takes to help early stage companies, and the strength of our community of mentors and experts. As an Actuator entrepreneur you should experience all of these, and they are your clue that the Actuator is indeed working.

Inventor Organizations

In this category I would include organizations like Universities, National Laboratories, Government development organizations or programs and the like. Obviously not all of these organizations, and obviously not every one to the same degree, but the generalization here is that these researchers look first to develop the best technology, then only later think about possibilities for commercialization. In some ways this, along with our strong national basic research capacity, has been the jewel in the crown of American global competitiveness for decades. But as budgets have consistently tightened and questions about Return on our research Investment have grown, this open-loop system that grew up in the aftermath of World War II may need some tweaking.

I would proffer three possible fallacies in this development approach in today’s environment. First is the belief that the quality of the technology is what drives the success of commercialization efforts. We evaluate a lot of companies for potential investment, and a common rule of thumb is that about 50% of an investment decision is made on the basis of the entrepreneurial team, perhaps 30% on the market dynamics (size, competition, path to market opportunities) and only the remaining 20% or so on the technology itself.

A second fallacy is that the researchers or developers know what the market wants; they are as a group incredibly smart and talented people who have relied on their judgement for success throughout their career. Like our entrepreneurs, they are almost always wrong with their first guess on what the market wants. This is why most companies in the early stages of development “pivot”, meaning substantially change something in their original product concept. One of the reasons that commercial markets are so successful is their relentless, continuous pressure to deliver, deliver ever better products, and deliver only what the market will pay for. Responding to this pressure is what makes companies continuously improve, and developing technologies in isolation from this pressure only delays the inevitable reckoning.

The third fallacy, somewhat related to the first two, is that what researchers and developers do is “innovation”, and innovation is what the market wanInnovationts. Jon Gertner in his great book The Idea Factory, about the operation of Bell Laboratories during the development of our national telecommunications network indicates the Bell Labs working definition, shown in the Figure.

What researchers and developers do is often Invention by this definition, but Innovation is really the end result of what we now call commercialization activities. Perhaps markets do want innovation, but it is important to be clear about what that means.

Is there a way to address these issues and improve the innovation outcomes for these Inventor Organizations? I believe the answer is “yes”. We are now exploring ways to connect our commercialization expertise directly to the research, inventions and entrepreneurs within these Innovation organizations. Demonstrating success in valuing IP, in business models that appropriately share the positive outcomes of commercialization, and in partnerships that overcome the biases that each side brings may well help improve the ROI for our Invention community.

Federal Government

The Federal government and its interactions with the R&D community may be in need of the biggest update. Many people point to the very cumbersome Federal Acquisition Regulations (FAR) as a road block to innovation. In fact, our experience is that the government probably has most of the legal authorities and mechanisms it needs to be much more effective as an R&D enterprise, but long-standing practices and cultural norms are really a much larger impediment.RandD spend corp gov

One issue is that the Government in many ways still acts as though it is 1950 when Government R&D spending was the dominant source of funding and the Government was large enough to constitute the primary market for innovative companies. This is no longer true, and in fact the relative market positions of the Government and commercial worlds have essentially reversed. The commercial world now spends twice as much on R&D as the Government, and represents a much larger market for innovative companies with more rapid paths to success.

A second recurring issue is Intellectual Property. Government encumbrance of small company IP in exchange for $50K or $100K development contracts makes those companies essentially uninvestable. Yet there are mechanisms in the Government contracting arsenal that do not require this encumbrance, and the value to the Government of locking up IP at such an early stage is minimal at best. So why does this practice persist?

Finally there is the structural problem. In the commercial world a path to market is critical. In the Government market, development support dries up around the SBIR Phase III point (working prototypes at some degree of maturity), followed by limited transition support to the uncertain market of large procurement programs. Why an uncertain market? Government program managers are incentivized to be risk averse, and new technology is almost never operationally robust when it is first introduced. The path to market for these large programs is most often through big systems integrators, and this is inherently risky for that precious IP. And, Government procurements are notorious for delays in awards, changes in scope and similar vagaries that can put a small company out of business long before a contract is ever awarded.

Here too there are ways to improve these outcomes. Certainly more support for transition programs that take interesting prototypes and help mature them would be a step in the right direction. The Governmenthas numerous test and evaluation capabilities that could be appropriately harnessed for this purpose, well within the limits of current contracting comfort zones. Adoption of more commercial-like practices such as those employed by some successful Government programs (In-Q-Tel, SBIR for example) can help get early market feedback and sufficient market competitive pressure to foster continuous evolution of interesting ideas. Increased use of staged awards such as SBIR, where only Phase I recipients are eligible for Phase II and so forth would help level the playing field for small companies, instead of so much of the innovation dollars going to incumbents working to develop ideas in-house with only limited external review and pressure.

There are others. NASA has placed much of its software in the open source domain, providing both valuable initial IP to innovators as well as fostering increased interaction between NASA and the innovation community. Our EMERGE program with DHS adopted a “commercial-first” approach, transitioning commercial technology into Government uses instead of trying to push Government-developed technology out.

Even the Chinese might provide an interesting model. Their “Made in China by 2025” initiative may sound like industrial policy, but seems to rely on commercial development of commercially viable products within broad sector definitions established by the Government. The implied quid pro quo is that the Chinese Government will then buy products from the best of those commercial companies.


So there it is, the 4 Part series on the Actuator, Innovation, and the various sectors of our economy that provide innovation. Improving success in this arena is indeed a wicked problem but there is room for substantial improvement simply by thinking about our collective goals and improving some of our innovation processes. Both our commercial and our national interests may be at stake.

Next Post: Smart City Actuator Focus Areas – Transportation

How Do We Know the Actuator is Working? Part 3 – Corporate/University Programs

Follow us @CITOrg or @dihrie or this blog for current information on the new Smart City Actuator.

Parts 1 and 2 of this series looked at methods for measuring the success of commercial and government programs for accelerating innovation. The other major sources of innovation in the U.S. economy are Corporate R&D efforts and University programs. How effective are these sources for driving innovation? It is hard to generalize, given the very large number of both Corporate efforts and University programs and the paucity of data, although the anecdotal data is mixed to negative.

It does seem as though our model of funding leverage and transition probabilities could be adapted to measure success here as well. For this analysis however, alternate data sources provide some quantitative clues that support the anecdotal data. On the Corporate side there are certainly large company innovation success stories. Apple for example has certainly had success with their early mass market personal computer, photo_mac84pand more recently with their mobile devices (iPhone, iPad). One of the more amazing companies in this regard has to be General Electric who, over the last 50 years (based on where their profits originate), has gone from a heavy equipment manufacturer to a financial services company to the current big data company that also happens to be one of the world’s leading manufacturers…and always as a market leader.

But the anecdotal and other data tell a different story. Many accelerator organizations around the country conduct essentially consulting engagements with large Corporations to help them engage tFailureshe innovation ecosystem. Data show that 86% of the original 1955 Fortune 500 companies no longer exist, and that 50% of the Fortune 500 in the year 2000 have also failed. The average life expectancy of all companies is only 12.5 years, and the failure rate seems to be increasing. According to Forbes, the reason “Why U.S. Firms Are Dying: Failure to Innovate

One solution might be for Corporations to invest more in R&D. Alas. <re/code> reports on a clever study from Bernstein Research analyst Toni Sacconaghi that looked at “historical R&D spending figures as a percentage of sales for publicly traded tech companies with market caps of $2 billion”Large Cap Stocks, then tracked their stock performance 5 years later when presumably the research might have paid off. The chart summarizes his data, and concludes that there is “no meaningful relationship between R&D spending and stock performance”. I had a hard time deciphering the chart, so decided to put it in a more graphical form, and insert a few trend lines. The results were actually worse than SaccoCorporate Optimum R&amp;Dnaghi was willing to conclude. Not only is the regression line negative…the more you spend the worse your results…the standard deviation decreases with spending, meaning that the likelihood of a poor result is much higher the more you spend. Looking at this chart as a non-financial person, it is not clear to me why anybody would invest in companies spending more than 20% of sales on R&D. As a side note, something really interesting seems to be happening around a magical number of 7-8% R&D spending, but I haven’t a clue what that might be…ideas welcome!

How about Universities? Many of them have Tech Transfer offices for their research-derived intellectual property (IP), and again there are certainly success stories in MIT/Harvard, Carnegie-Mellon, Stanford and others. Anecdotally however these offices, many based on a traditional licensing model, have overall not been incredibly successful. Two reasons are often given: first, IP transfers much more effectively via people than via paper so licensing models without researcher support are not effective; and, second, some Tech Transfer offices like to value the IP on the basis of research dollars spent, not on market value. As one of my colleagues put it “when I have these discussions the Tech Transfer Office will often tell me the value of the IP is $10M or something; I tell them it’s worth zero unless somebody takes it to market, and we negotiate from there.” Fortunately some of the more forward-looking Universities are starting to migrate towards a shared risk/reward model where the value of IP is set by marketplace outcomes.

Is there an explanation for this phenomenon? One possible answer lies in where the funding for research comes from, and where it goes. Again an obscure chart, this one from NSF.Funding sources After staring at this one for a while what I really wanted to know was net funding flows. For example in Basic Research the Federal Government funds about 60% and only conducts about 10%…where does the other 50% go? Mostly to Universities, which fund 10% and use more than 50% of the Basic Research dollars. OK, easy enough to turn into a graph, where the nodes are the sources, scaled to size by percentage Research fundingof funds sourced, and the graph edges are the net flows, also scaled to size. By applying some awesome powerpoint skills and a garish color scheme, the Basic Research picture looks like this. Half of all research dollars go from the Federal Government to University-based research, with a small additional amount funded by Industry, totaling about 2/3 of all research performed.

Now applying the same analysis to the Development portions of the NSF chart yields the following.Development funding Here, almost ninety percent of all Product Development activity is funded and performed by Industry, with some support from the Federal Government, while Universities are almost invisible. No wonder there is a bit of a disconnect; Universities apparently are not at all focused on commercializing their research, if the funding numbers are to be believed. One last chart provides the summary. Looking at the per capita number of startups based on University-developed IP the numbers have been dropping for a while. More to the point, the numbers are low. For Virginia, for example, these numbers equate to about 20 startups per year. Our cybersecurity accelerator, MACH37, evaluates more than 100 companper capita unievrsity startupsies per year to select a dozen participants just in the area of cybersecurity. Numbers are similar for most venture investors, with only single digit percentages of the number of deals reviewed actually resulting in investment. Thus for Virginia this may equate to one or two investable companies per year, based on University-generated IP. To be fair, this probably underestimates the amount of entrepreneurial activity generated by University programs and proximity, but is probably reasonably accurate in terms of the Institutional success, based at least on the anecdotal evidence.

It is clear now, having looked at innovation and technology transfer across commercial accelerators, the Federal Government, Corporations and Universities that successful innovation is one of those “wicked problems.” While there are successes across each of these domains, there are no magic bullets, no guaranteed approaches for innovation. So, how do we know our Smart City Actuator is working? And, are there ways to make this entire national research enterprise more efficient? We will explore those questions in Part 4 of this series.

Next (Thursday 3/23): How do we know the Actuator is Working? Part 4 – Synthesis and Policy