As enterprises race to scale AI, rising GPU bills and data-residency worries under the DPDP Act are forcing a rethink. Is a frontier model really needed for every task? Anuj Gupta, Founder & CEO, KiteFishAI, says no. His startup builds small language models, trained on Indian languages and domain data that can run on a laptop or on-premises CPU. In this conversation with CIO&Leader, Gupta explains when a 0.6B-10B model beats a general-purpose LLM, why BFSI, healthcare and pharma should avoid foreign-hosted APIs, how drift is monitored after deployment, and what India still needs to build homegrown foundation models. He also credits government GPU subsidies, while flagging a talent and clean-data gap.
CIO&Leader: What changed in enterprise AI, and when should a CIO choose a 0.6B-10B model instead of a frontier LLM?
Anuj Gupta: The world is moving very fast; every industry is using AI, and now they have to. When they have to use AI and the return on investment is very low due to high costs, the only way around this is to use a model that serves their purpose for a particular task, not a frontier model for everything. Anything you would need a frontier model for—for example, if you have to go to a pan shop to buy a cigarette or something, you would not go maybe through a car, or you would not go through a metro; you would step out of your office, go there, buy something, and come back. That is the easiest and the best way. We’re saying that if you use a specific task, you use specific models that will give you a high return on investment, and they can switch to an MBD model with 0.6 billion parameters. They can do their task; they can move to a lighter-weight text model that can do it, or use a domain-specific finance model. When they need to perform a heavy task, maybe image generation or video generation, go to a frontier model, get your things done, and then return to these smaller models that can do your task. So that is how, and that is when they have to use smaller models so that their cost is also very low, and at the end of the day, they are also using AI.
CIO&Leader: Where do small language models match or beat general-purpose LLMs, and where do they still fall short? Can you share a case where the right-sized model won?
Anuj Gupta: For domain-specific work, for example, if you are talking about a BFSI model, okay. If you go to a BFSI model, like if you go to Claude and you ask, like, what are the regulations of the RBI in India, it may give you a very generic answer. Why? Because it also knows US guidelines, it also knows European guidelines, Japan, China, every other country’s guidelines it has, okay. But if you pick up a small model trained only on the RBI guidelines in India, which are updated weekly or monthly, it will give you a better answer. Why? Because it knows only that guideline, and it has to answer only from that guideline, okay. So that is where you should be using a model, a domain-specific model. To answer your second question, like at what size should we define, like we should use this model or that model, see there is no particular category. The number of parameters only increases the amount of accuracy a model can serve, okay. If a 3-billion-parameter model can’t answer all BFSI-related questions, move to a 5-billion-parameter model, then to a 7-billion-parameter model. But to a greater extent, a 7-billion-parameter model won’t be able to answer all your questions. But at the end of the day, you also have to keep in mind that a 7-billion-parameter model will take a little longer than a 3-billion-parameter model because GPU requirements increase.
So that is another factor that we have to keep in mind because the cost is also increasing. But there is just a very fine line between which model you should use for which type of activity. It usually comes from the fact that, over time, we conclude that maybe we can limit ourselves to a 5-billion-parameter model and it will be able to serve everything we need. But there is no clear demarcation, to be honest, as to which parameter you should use for which kind of task.
CIO&Leader: What risks do global LLM APIs create for data control, vendor dependence, and DPDP compliance?
Anuj Gupta: DPDP is very clear that the data should remain in India. That is what DPDP says: Indian consumer data should remain in India. It should not go outside India. And it is fair. Why? Because all of these models that have been created mostly have been created using Indian data. Why? Because India was not very much; India never used to control the data we produce, or that goes to these companies. So India never used to control them. And these models are very data-hungry. The more data you give them, the better they will become. India has been, and remains, a playground for all these models to date, and they are still collecting data there. Especially for companies in the BFSI, healthcare, and pharma sectors, this DPDP Act is highly relevant because the data there is highly proprietary. These companies do not want their data to leave their systems. For them, the use of models must continue. Even if the model is on an API, it should be hosted in an Indian cloud and not on an American or any other country’s cloud. That is very important. The moment it leaves India, you lose control. And once you lose control, it can be used for anything and everything. Even if they say they will not use the data for model training, they will still use it because there is a clause that every country must abide by, which requires you to keep it for 30 days whenever a customer sends your data. You are allowed to keep it for 30 days to check whether they have not used it to gather wrong information. For that purpose, they will keep that data for 30 days.
Now, within those 30 days, for example, I will use a random example: person X has used Claude to find out whether I can make a bomb. Now till the time the person X name is there in that chat information- that question- that is person data. The moment you remove that PII information, which is the name of the person, it becomes generic data. That X person, like somebody, asked for how to create a bomb. This becomes generic information. You only need to remove PII from that data; once you do, it is no longer your data. It becomes generic data that can then be used for training. Ultimately, at least companies in the BFSI, pharma, and healthcare sectors should not be using APIs hosted in American or any other cloud.
CIO&Leader: What does “Sovereign AI” mean for KiteFishAI in practice, and what proof should enterprises ask for?
Anuj Gupta: We are specifically building models for Indian companies. We are training our models on Indian languages, and what it means is that, as I mentioned earlier—for example, an example I gave earlier—our model will be able to answer a question related to RBI guidelines better compared to Claude or any other model because we have created our model with that as a base. So our information is there in the model itself, and it keeps getting updated every month. So even if RBI guidelines are updated monthly, our model is updated monthly as well, bringing you the latest information every day. Compared to a Claude that will update its knowledge maybe once in three months or once in four months because the model over there is very heavy, very, very, very, very heavy compared to us. It is a thousand times heavier than us. It is not easy for them to update their knowledge every month or every two months. It has to be every three months or every four months. By that time, if something crucial for the industry has been released, companies cannot wait three or four months to get that knowledge. So we are working very fast on this. Kitefish is working very quickly to build small language models that serve the enterprise’s purpose. They get the latest information every time, and because our models are very small, they also get updated every month or every three weeks or every four weeks.
So we have a very fast refreshing cycle because of which the knowledge is updated very quickly.
CIO&Leader: How do specialized private models compare economically with per-token pricing for large models, and when does the advantage become clear?
Anuj Gupta: The definition of a small language model and a large language model is anything below… To be honest, there is no standard industry-wide terminology for SLM and LLM. But to some extent, everybody follows this: anything below a 20-billion-parameter model is a SLM, small language model. Anything above a 20-billion-parameter model is a large language model, and anything beyond a 1,000- or 1-trillion-parameter model is a frontier model. So this is a standard that everybody uses. To answer your question about how KitefishAI is working on this: running a small model requires less GPU. For example, if we are running a 1 billion parameter model, it would only require 4 GB GPU memory, okay, which is very, very, very little. It can also run on your systems, which means the cost is very low. If you are running it on your system, which is available to you 24/7, the cost is very low. But running a frontier model or a large language model would definitely require a GPU. Even one H100 would cost you somewhere around 1 lakh rupees per month. So you are increasing that cost by multiple folds. So that is the difference that we bring in through small language models that you can also run them on your system and you can also have them, like they can serve you whatever needs are on your system.
CIO&Leader: How did you decide what to build across reasoning, OCR, embeddings, speech, and vision-language models, and how do you ensure domain accuracy?
Anuj Gupta: We have multiple teams working on multiple domains. That is how we keep ourselves updated. But in terms of modalities like speech or whether it’s embedding or an OCR model. Again, we have multiple teams, but it really depends on what clients are asking for. In India, we have seen two major modalities in use. One is speech. The other one is text. So our primary focus is on speech and text. We have built OCR models and other model sets, but our primary focus is speech in India and text.
CIO&Leader: In BFSI, healthcare, and pharma, how do you validate domain-specific models and address hallucination, auditability, explainability, and final sign-off?
Anuj Gupta: We have a platform called Prism that we ship with our models. The Prism platform helps you check whether your model is drifting now or then. The comparison is that whenever we release a model, we release it with a certain accuracy. For example, we release it with 90% accuracy on the benchmarks. Benchmarks define your model’s accuracy. We release our model’s accuracy, which is around 90%. Over time, as the company’s data is updated and the model does not reflect these changes, its accuracy may drift. How do you measure it? Our solution, Prism, which we ship with our model, helps us see whether the model was able to answer earlier with a better response and is now unable to answer, which helps you identify the reason for the model drift.
Once there is a model drift below a certain threshold- for example, we release the model at 90% accuracy, and now the model has dropped to 80% accuracy- we will again fine-tune. We will ship the model back to the company so they can maintain accuracy and get the answers correct. That is the first thing. Now, on the observability side, multiple factors contribute. For example, how many people are using it, how much cost people are spending on using these models, and everything. Again, Prism helps you identify usage by sector within the company. It will be able to tell you, send you a notification saying that this department is using these models at this much rate and at this much cost, and then you can probably manage it internally, like you can put conditions that the department can use only this much amount of model and everything. So, the Prism solution we ship with our model helps you manage model observability and security.
CIO&Leader: Air-gapped and on-premise deployments are seen as hard and costly. What does it take to run your models there, and what do customers usually underestimate?
Anuj Gupta: Air-gapped is a misconception. Air-gapped or on-premises will cost you. It is only applicable while you are building an LLM. But as I mentioned, since our models can also run on your laptops, the cost is significantly lower. You don’t really need a GPU every time to run a model, which means running on your laptop is air-gapped, and running on-premises on a CPU is, again, on-premises. So, really the cost is not very high when it comes to running an SLM. When you build and run an LLM, the cost becomes very high. Yes, but because we are in the SLM domain, which is a small language model domain, the cost is very, very low compared to LLMs.
CIO&Leader: How much data and expertise does an enterprise need to fine-tune effectively, and how do you manage data leakage, model drift, and maintenance across specialized models?
Anuj Gupta: Somewhere around 1GB or 5GB of data is good enough to fine-tune a model, an SLM. It can be done once every two months; that’s sufficient to keep the model as accurate as possible. How do we do it? Again, using a prism model, we monitor model drift and check whether it falls below a certain threshold; if it does, we fine-tune the model. But around 1-5 GB of data would be needed again for model fine-tuning to restore the same level of accuracy.
CIO&Leader: India has the talent and data but lags in foundation models. What does it need most: compute, funding, datasets, policy support, or adoption of homegrown models?
Anuj Gupta: I was having a conversation with one of the IIT professors three months back, and they told me specifically that, Anuj, you can hire students from our college, PhD students in AI, but you would have to teach them how to build a model. So, there is a huge gap, to be honest. Yes, data is available, but how to consume it and how to build a model architecture are huge gaps. We can have that level of competency, or we can have that level of skill built only over time. It is not an overnight job where I ask 10 people to sit under me, teach them, and tomorrow they become AI experts or AI model experts. It can only happen over time. You have to spend time learning the mathematics around it. You have to understand what goes into data, how to build a model, what you mean by model architecture, what you mean by open-weight models. There are so many things that you will have to learn. A period of two or three months would get you started, but it is time-consuming. It cannot happen overnight.
Really, India will have to spend time teaching people, and that is one thing. If you ask me, what should the government do? The government is really doing its bit by providing GPUs at a very low cost, and no denying that fact. The government of India, especially the Ministry of IT, has done an excellent job in providing GPUs at subsidized rates, which is really commendable. If this had not been there, companies like KitefishAI and many others would have struggled to exist. That is the first thing. The government can also provide the GPUs at a subsidized rate and provide a pool of clean data. Cleaning data is a bigger mess. Even if you put 100 people on it, data exists independently, but determining what kind of data to use and which data language to use requires a huge amount of investment. That is another area where the government of India can allocate its resources. I think the government of India is doing a really good job, to be honest, on that side. They can still do one or two things, but they are doing a good job.