Why does code testing begin at startup? Nova AI uses more open-source LLMs than OpenAI
Human nature universally dictates that code developers shouldn’t be the ones testing their creations. To begin with, most of them genuinely dislike that task. Second, the people who perform the task shouldn’t be the ones who verify it, just like in any decent auditing system.
It should come as no surprise, then, that a growing number of generative AI businesses have made code testing—including usability, language- or task-specific tests, and end-to-end testing—a priority. TechCrunch features new ones every week, such as QA Wolf (funded $20 million), CodiumAI (raised $11 million), and Antithesis (raised $47 million). Additionally, new ones—like Momentic, a recent Y Combinator alumnus—are always developing.
Another is the one-year-old Nova AI firm, which was founded by an Unusual Academy accelerator graduate and has raised a $1 million pre-seed investment. Founder and CEO Zach Smith tells TechCrunch that the company is trying to outperform its rivals with its end-to-end testing tools by flouting several Silicon Valley norms around how businesses ought to function.
While the typical Y Combinator strategy is to start small, Nova AI is targeting mid-size to large businesses that have urgent needs and complicated code bases. While describing them as primarily late-stage (Series C or beyond) venture-backed firms in e-commerce, finance, or consumer products and “heavy user experiences,” Smith declined to name any customers utilizing or testing its platform. These features’ downtime is expensive.
The technology at Nova AI goes through the code of its clients to create tests automatically using GenAI. It is specially designed for situations that use continuous integration and delivery/deployment (CI/CD), where engineers are continuously integrating small pieces of code into their production code.
The experiences Smith and co-founder Jeffrey Shih had as engineers at large tech businesses served as the inspiration for Nova AI. Smith was a member of the Google team before leaving to work on cloud teams that assisted clients in utilizing a lot of automation technology. Prior to joining Meta, Shih held positions at Microsoft, Unity, and a unique AI specialty involving synthetic data. Since then, Henry Li, an AI data scientist, has joined as a third co-founder.
Another rule that Nova AI is not abiding by is this. Whereas a plethora of AI firms are developing on top of OpenAI’s industry-leading GPT, Nova AI is making minimal use of OpenAI’s Chat GPT-4. No consumer information is supplied to OpenAI.
Businesses still do not trust OpenAI, despite the company’s assurances that data from those with paid commercial plans is not used to train its models, according to Smith. Large businesses often say, “We don’t want our data going into OpenAI,” according to Smith.
Not just the technical departments of big businesses have this feeling. Many lawsuits from people who don’t want OpenAI to use their work for model training or who think their work ended up in its outputs without authorization or payment are being repelled by the company.
Rather than developing its models, Nova AI mostly depends on open-source models like Llama, created by Meta and StarCoder (from the BigCoder community, which ServiceNow and Hugging Face produced). Although they haven’t used Google’s Gemma with clients yet, Smith notes that they have tested it and “seen good results.”
He mentions, for example, that OpenAI provides vector embedding models. Text segments are converted into numbers using vector embeddings, allowing the LLM to carry out a variety of tasks, like grouping the comparable text segments. Instead of using OpenAI’s embeddings, Nova AI leverages open source on the source code provided by the customer. It goes to great pains to avoid sending any client data into OpenAI, and it solely uses OpenAI tools to help it develop some code and do some labelling tasks.
In this instance, Smith said, “we deploy our own open source embedding models instead of using OpenAI’s embedding models so that we aren’t just sending it to OpenAI when we need to run through every file.”
While businesses can relax knowing that their consumer data isn’t being sent to OpenAI, Smith has discovered that open-source AI models are more than adequate for completing focused, niche jobs and are also less expensive. They are effective in this instance for writing exams.
“When you go really narrow, the open LLM industry is really proving that they can beat GPT 4 and these big domain providers,” the speaker stated. We don’t need to give you some elaborate model that can predict what your grandmother would like for her birthday. Correct? We must draft an exam. That’s all there is to it. Our models have been adjusted especially for it.
Additionally, open-source models are developing swiftly. For example, Meta has released a new version of Llama that is receiving praise in the tech community and could persuade other AI businesses to consider OpenAI substitutes.