Community Bonding Period

May 1 to May 24, 2026

The community bonding period is supposed to be about getting familiar with the codebase, setting up your environment, and getting to know your mentors and fellow contributors before coding starts. That is exactly what I worked on these 3 weeks. There was also a major change with respect to what I would consider my baseline on going forward that changed the direction of the project in a good way.

Getting Set Up

The first thing I did was set up my local development environment and start experimenting with the core APIs the project depends on.

I pulled qwen3:8b locally via Ollama and ran a quick check to confirm it responds to prompts correctly. This model is the backbone for most of my other tasks down the line, the Planner, Entity Linker reasoning, and Answer Generator nodes in the architecture I proposed.

The first real finding came immediately. I had assumed the DBpedia Lookup API returns JSON by default. It does not. It returns XML unless I explicitly set the Accept header to application/json. A small thing but exactly the kind of detail I figured would break the pipeline silently if I had assumed rather than test.

The SPARQL endpoint tests were more interesting. I ran two queries for New York City's population. The first used dbo:population which is what a language model would naturally guess:

{ "bindings": [ ] }

Empty. No error, and no warning.

The second used dbo:populationTotal which is what DBpedia actually stores it as:

{ "pop": { "value": "8804190" } }

8,804,190. Correct.

This is the exact failure mode the entire agentic architecture is designed to handle. A one-shot system returns nothing and the user never knows why. I also tested the Einstein birthplace query and the Berlin area query. The birthplace query worked fine. The Berlin area query failed silently because the correct predicate is dbo:areaTotal, not dbo:area. Same pattern.

On May 4th I was added to two Slack groups, a general GSoC 2026 DBpedia group for all selected contributors and a dedicated project group with my mentors Tommaso Soru, Ronit Banerjee, Abdulsobur, and Gandharva Naveen. Introducing myself to the team made the project feel real and felt like an accomplishment in itself.

Going Through the NSpM History

My proposal mentioned reproducing baselines from the gsoc/anand and gsoc/zheyuan branches of the neural-qa repository. I cloned the repo and checked all six contributor branches. Both of those branches require TensorFlow 1.x, which is not compatible with modern macOS (the system I will be using throughout my GSoC). Rather than spending bonding period time on dependency issues, I went further and set up Mehrzad's 2023 branch instead, which uses a modern transformers stack.

I set up a Python 3.9 virtual environment, found and fixed two missing dependencies in his requirements.txt (pyarrow and importlib_metadata were absent, causing import errors on a clean install), patched a CPU compatibility issue in merge_model.py which hardcodes cuda:0 as the default device, and successfully merged his CodeGen 350M checkpoint with the base model.

I then extracted the full evaluation results from his CSV files. I wanted to have a look at these numbers since they were never published in Mehrzad's blog:

One thing worth noting about these numbers. BLEU measures string similarity between the generated SPARQL and the gold standard query. It does not check whether the query actually returns the correct answer. A query can be structurally different from the gold standard but semantically identical and score near zero on BLEU. The system I am building will be evaluated on F1 against actual answer sets, which is a stricter and more meaningful metric.

Discovering the 2026 Baseline

About halfway through the bonding period, I messaged my mentors about a few dependency issues I found in Mehrzad's branch, which I have also highlighted in the above section. One of my mentors, Tommaso Soru, replied with something that significantly changed the direction of my preparation. He and my other mentor Abdulsobur had already built and submitted a full agentic KGQA system to the Text2SPARQL 2026 challenge in April, and he shared the repository with me as the new baseline.

The system, called agentic-kgqa, is a pure Python pipeline without any agentic framework. It does entity linking via a shared Redis server preloaded with DBpedia surface forms, ontology lookup via word2vec embeddings over DBpedia properties, LLM-based SPARQL generation, and a self-correction loop that retries up to two times on failure. It supports multiple LLM backends through OpenRouter.

They submitted two versions to the challenge, one powered by Claude Sonnet and one by Qwen 3.5 122B. The official results on the DB26 DBpedia track:

These are the real baselines for this summer. Not Mehrzad's BLEU scores from 2023.

I spent the rest of the bonding period getting the system running locally. This involved downloading the ontology vectors file Tommaso shared, pulling the DBpedia 2015-10 dataset files, loading them into a local Oxigraph instance running in Docker, and starting the FastAPI server. After some dependency debugging and a lenient reload of the mappingbased_objects file to get past a parse error, the full pipeline worked end to end.

I tested it on a few questions. The Keanu Reeves birthplace query went through all five stages correctly: question analysis, Redis entity linking, ontology embedding lookup, SPARQL generation, and execution against the local Oxigraph endpoint. It returned Beirut on the first attempt with no revision needed.

Community Call

We also had a community call on May 22nd with all the accepted GSoC 2026 DBpedia contributors and mentors. Julia, followed by Tommaso, from the DBpedia Association kicked things off with a briefing on how the program works, shared some interesting statistics about DBpedia's GSoC history like how many students were accepted each year and how many continued contributing after GSoC ended, and gave us tips on having a successful summer including communicating regularly with mentors and documenting progress through blogs. Each contributor then introduced themselves and their project in about a minute. There was a Q&A session after where mentors answered questions from the group. We ended with everyone turning their cameras on for a group photo.

What's Next

Coding starts May 25th. The first thing on the list is understanding exactly where the current system fails and why, using the per-question breakdown from the Text2SPARQL results. The docs/analysis.md file in the repo documents nine prompt versions and concludes that 22% structural accuracy appears to be the ceiling for prompt engineering alone on this task and model combination.

My goal this summer is to push past the 32% F1 baseline by adding a FAISS vector fallback for entity linking on ambiguous mentions, improving the ontology exploration step to reduce predicate hallucinations, and running a full evaluation against the QALD-9-plus benchmark which the current system has not been tested on.