New method improves specification generation for OS kernels with AI
Researchers have developed BODHI, a new method that improves how large language models (LLMs) generate formal specifications for system calls in operating system kernels, achieving up to 74.69% in Pass@1.
What happened?
Researchers have introduced BODHI (OS Kernel Specification Inference), a method to improve the generation of formal specifications for system calls in operating system kernels using large language models. The method utilises a domain-knowledge prompt that supplements standard few-shot prompts with a structured C-to-Python translation guide. This guide covers 15 categories of domain-specific translation patterns and organises the translation by separating concerns, where pre- and post-conditions are handled as distinct categories.
Key facts
| Metod | BODHI (OS Kernel Specification Inference) |
|---|---|
| Benchmark | OSV-Bench |
| Antal uppgifter i OSV-Bench | 245 |
| Tidigare Pass@1 | 55,10% |
| BODHI Pass@1 | 74,69% |
| Antal modeller utvärderade | 9 från 6 leverantörer |
”We propose a domain knowledge prompting method (BODHI), which augments the standard few-shot prompt with a structured C-to-Python translation guide covering 15 categories of domain-specific translation patterns.”
”Inspired by Structured Chain-of-Thought (SCoT) prompting, the guide organizes translation by separation of concerns, addressing pre-condition extraction and post-condition generation as distinct categories.”
”Evaluated on nine models from six providers (Anthropic, Mistral, Amazon, DeepSeek, Meta, Alibaba), cov(—) The best reported Pass@1 is 55.10%.”
Why it matters
Manual creation of precise specifications for operating system kernels requires deep domain expertise and is time-consuming. By automating the process with LLMs, the development time for formal verification of OS kernels can potentially be reduced. The new method shows a significant improvement over previous benchmark results, indicating a step forward in overcoming challenges within software verification.
Who is affected?
The method impacts researchers and developers within formal verification and operating systems. Companies such as Meta, Amazon, Alibaba, Anthropic, and Mistral, whose models were evaluated, are also directly affected. Improved specification generation can facilitate the development of more secure and robust operating system kernels.
What else you should know
BODHI was evaluated on nine different LLMs from six providers. The results showed that the new method achieved a Pass@1 of 74.69% on OSV-Bench, a significant improvement over the previously best reported result of 55.10%.
Quick answers about this story
Vad har hänt?
När hände det?
Varför spelar det roll?
Vilka bolag berörs?
The link opens in a new window and leads to the publisher's own site.
Källan har spårats automatiskt från utgivaren via Aheadlines signalkedja.
AI-verktyg i artikeln
Topics
Get similar news straight to your inbox
The reader's room
Send in a question or an addition. The newsroom reads everything before it's published and replies when relevant. No AI-generated text – just people.
Sign in to submit a comment or question.
Read the article through your role
- Decide whether this affects strategy over 6–12 months or is just noise.
- Discuss with leadership: do we own the right question or does ownership need to move?
- Ask: what risk are we taking by NOT acting on this this quarter?
Generated angle — not editorial analysis of "New method improves specification generation for OS kernels "