Migrating LlamaIndex ServiceContext to Settings

ServiceContext was the object you passed everywhere to say which model, which embeddings and which chunk size. It was deprecated in the 0.10 line and removed in 0.11, replaced by a global Settings object. Most of the rewrite is one-for-one. The part that is not is the reason the migration is worth understanding rather than pattern-matching.

The message, and which one you got

Two different strings, and they tell you how far past the change your install is.

• A deprecation, worded along the lines of ServiceContext is deprecated, please use `llama_index.settings.Settings` instead, or pass in modules to local functions/methods/interfaces. You are on 0.10.x. Everything still works.
• ImportError: cannot import name 'ServiceContext' from 'llama_index.core'. You are on 0.11 or later, where the 0.11 release notes record the removal. Nothing runs until the rewrite is done.

A third string tends to arrive at the same time and is a different change: cannot import name 'ServiceContext' from 'llama_index', with no .core. That is the 0.10 namespace split rather than the ServiceContext removal — the top-level package became a thin shell and the framework moved to llama_index.core, with each integration in its own distribution such as llama-index-llms-openai and llama-index-embeddings-openai. If you are on 0.9 you are doing both migrations at once; do the namespace one first, because otherwise every error message you read is about the wrong problem.

The rewrite

Before, on 0.9 or early 0.10:

from llama_index import ServiceContext, VectorStoreIndex, SimpleDirectoryReader
from llama_index.llms import OpenAI
from llama_index.embeddings import OpenAIEmbedding

service_context = ServiceContext.from_defaults(
    llm=OpenAI(model="gpt-4o-mini", temperature=0),
    embed_model=OpenAIEmbedding(model="text-embedding-3-small"),
    chunk_size=512,
    chunk_overlap=64,
)

documents = SimpleDirectoryReader("./docs").load_data()
index = VectorStoreIndex.from_documents(documents, service_context=service_context)
engine = index.as_query_engine(service_context=service_context)

After:

from llama_index.core import Settings, VectorStoreIndex, SimpleDirectoryReader
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding

Settings.llm = OpenAI(model="gpt-4o-mini", temperature=0)
Settings.embed_model = OpenAIEmbedding(model="text-embedding-3-small")
Settings.chunk_size = 512
Settings.chunk_overlap = 64

documents = SimpleDirectoryReader("./docs").load_data()
index = VectorStoreIndex.from_documents(documents)
engine = index.as_query_engine()

The mechanical rules: ServiceContext.from_defaults(k=v) becomes Settings.k = v, and every service_context=service_context argument is deleted rather than renamed. There is no set_global_service_context equivalent to call, because Settings is already the global — assigning to it is the whole registration step.

The attributes carry over with the same names. The Settings documentation lists llm, embed_model, node_parser, text_splitter, chunk_size, chunk_overlap, transformations, callback_manager, tokenizer, context_window and num_output. One behavioural detail that catches people: the members are lazily instantiated, so setting nothing and calling an index still works and quietly uses a default model. An AuthenticationError from a provider you did not think you were using is usually this.

The part a global cannot express

A ServiceContext was a value. You could have two of them and pass a different one to each index — a cheap embedding model for a large low-value corpus, an expensive one for the corpus that matters, two chunk sizes for two document shapes. A global cannot hold two answers at once, and this is the one place the migration is not mechanical.

The replacement is local overrides, which the documentation is explicit about: the modules that used to take a service context take the individual pieces instead.

# per-index override
index = VectorStoreIndex.from_documents(
    documents,
    embed_model=cheap_embed_model,
    transformations=[custom_splitter],
)

# per-query-engine override
engine = index.as_query_engine(llm=expensive_llm)

So the translation for a codebase with several service contexts is: pick the one that applies most widely and make it Settings, and turn every other one into explicit keyword arguments at the call sites that used it. That is more typing than before, and it is also the reason the change was made — with a service context threaded through five layers, the model actually in use at the bottom was a question you answered by reading the call stack.

There is also an asymmetry in when each setting is read, and it survives the migration unchanged because it was true of ServiceContext too. Settings.llm is consulted at query time, so changing it affects the next question you ask. Settings.chunk_size and Settings.chunk_overlap are consulted during ingestion, so changing them after an index exists does nothing at all to that index. Somebody tuning chunk size by editing the value and re-running a query is tuning nothing; the setting only takes effect when documents are parsed into nodes again, which is the general point made in the chunking treatment.

The failure that waits until query time

The dangerous version of this migration is the one where the index is persisted. An index stores vectors produced by whatever embedding model was configured when it was built. Nothing in the stored index forces the same model to be used at query time, and nothing checks.

If a service context set one embedding model at build time and your new Settings defaults to another, one of two things happens. Either the dimensions differ, and you get a loud shape or dimension error from the vector store — annoying, immediate, easy. Or the dimensions happen to match, the query embeds into a different vector space, and retrieval returns confidently wrong neighbours with no error at all. Two models with the same output width are entirely capable of this, which is why matching dimensions is not evidence of matching models.

The check is cheap. Before the migration, record which embedding model built each persisted index and store it beside the index. After, run a handful of known queries whose correct top result you already know. If the top results changed, you have loaded an index under the wrong embedding model, and the fix is to set Settings.embed_model back to the one that built it or to rebuild.

The other removals in the same window

• LLMPredictor — removed alongside ServiceContext. The LLM object is used directly in its place, so a LLMPredictor(llm=...) wrapper is simply deleted and its inner model assigned to Settings.llm.
• PromptHelper settings — the token-budget knobs it held, context_window and num_output, are attributes on Settings directly.
• Integration packages — every import of the form llama_index.llms.X or llama_index.embeddings.X requires its own installed distribution. A ModuleNotFoundError here is a missing requirement, not a wrong path.

Doing it

1. Record the current versions and, for every persisted index, the embedding model that built it. This is the piece of information the migration can destroy and nothing else recovers.
2. If you are below 0.10, do the namespace move first: llama_index to llama_index.core, plus the integration distributions. Get that green before touching configuration.
3. Find every ServiceContext.from_defaults in the tree. If there is exactly one, this migration is ten minutes. If there are several, list which call sites each reaches — that list is your override plan.
4. Assign the dominant configuration to Settings at application start, once, in the module that owns startup rather than at import time in several places.
5. Delete every service_context= argument, and add local llm=, embed_model= or transformations= arguments wherever the deleted context differed from the global.
6. Re-run the known-answer queries from step one against every persisted index and compare the top results before declaring the migration finished.