Custom GPTs and Claude Projects: Where Upload-and-Ask Stops Working

Can a Custom GPT replace a document-search deployment? For a small, stable folder read by a handful of colleagues, yes — and you should start there. It stops working when the library gets large, when the answers have to be checkable by someone who was not in the room, when different audiences need different shelves, or when the answers leave your company. Those four thresholds, not file-count limits, are what actually forces the move.

Almost every organisation that ends up with a document-grounded AI deployment started by uploading a folder to ChatGPT or Claude and being pleasantly surprised. That is the correct first step. It is cheap, it takes an afternoon, and it tells you whether anyone actually wants the thing.

This article is about the second step: the specific points at which upload-and-ask stops being enough, why each one is structural rather than a missing feature, and what you have to own instead. We build Edtek Chat, which is one of the things you might buy at that point — so treat our framing as informed by operating experience, not neutral. The thresholds below are the ones we watch customers hit, in the order they usually hit them.

The four thresholds

Folder of files ──▶ Custom GPT / Project ──▶ works fine
                                              │
                    ┌─────────────────────────┼─────────────────────────┐
                    ▼                         ▼                         ▼
            1. Library grows        2. Answers get audited     3. Audiences diverge
               retrieval quality       "where does that            one brand, one shelf
               becomes the product      come from?"                stops being enough
                                              │
                                              ▼
                                     4. Content or answers
                                        must not leave

Threshold 1 — the library outgrows the context

A Custom GPT or a Project keeps a modest number of files and searches them with a retrieval layer you do not configure. At small scale this is invisible, because almost anything retrieves the right passage out of thirty pages.

At two thousand pages it stops being invisible. Retrieval quality becomes the product, and retrieval quality is a set of decisions: how documents are split, how wide the search runs before it starts discarding, how close a passage has to be before it counts, whether a second pass re-ranks what came back, and how much of one answer a single long document is allowed to supply. In a hosted assistant those decisions have been made for you, uniformly, for every customer. You cannot see them and you cannot move them.

That matters because the right settings for a shelf of dense statutes are not the right settings for a shelf of long research papers. Statutory text is repetitive and formulaic — similarity scores cluster; research prose is varied and discursive — they spread out. A threshold tuned for one silently over-filters the other. We describe the specific dials in what you can configure.

Threshold 2 — someone asks where the answer came from

This is the threshold that arrives soonest in regulated work and hurts most.

Hosted assistants do cite. What they typically give you is a file-level reference and a snippet — enough to know which document was involved, not enough to check the claim without opening the document and reading. In practice almost nobody opens the document, which means the citation is performing reassurance rather than verification.

The distinction that matters is whether a reader can go from a sentence in the answer to the passage that supports it in one click, and whether a citation can be wrong in a way the system cannot detect. There are three separable properties worth asking any vendor about:

• Can a citation point at something that was never retrieved? In a system where citation numbers are handed out by the retrieval step rather than written by the model, a reference to a passage that does not exist can be stripped before it is ever rendered. Where the model writes the citation text itself, it can invent one.
• Does the reader land on the page, or on the file? Page-level references require the ingestion pipeline to have tracked page geometry. It is an architectural decision made long before the question is asked.
• What happens when nothing matches? A system that always produces an answer is telling you it has no failure mode. That is not reassuring.

We go through all three in how grounded answers work, including what is enforced in code and what is merely made unlikely.

Threshold 3 — one assistant becomes several

The third threshold is organisational rather than technical. A publisher wants one assistant per imprint. A firm wants one per practice group. An association wants a public one and a members-only one, on overlapping but not identical material.

In a hosted assistant, each of those is a separate Custom GPT with its own separately uploaded copy of the files, its own separately pasted instructions, and no shared notion of a house standard. Update a policy document and you update it in six places. Change your wording for “we could not find that” and you change it in six places, from memory.

What replaces it is a configuration layer: one installation, many deployments, each with its own name, mark, palette, prompt, model, retrieval settings, members and shelf — and settings that fall through from the deployment to your global defaults to the product’s own, so a house rule is set once. That is the substance behind the phrase “white-label,” and it is worth asking to see the admin screen rather than the marketing page.

Threshold 4 — the content cannot leave

The last threshold is the one procurement raises. Sometimes it is regulatory, sometimes contractual — a licensing agreement that does not permit the text to be processed by a third party — and sometimes it is simply that the board has said no.

The relevant question is not “is the vendor secure,” it is “whose account is this running in.” A deployment provisioned into your own cloud account, as infrastructure-as-code you can read, with the database and object storage in your subscription, answers a different question than a hosted product with a good security page. On some clouds the language model can sit inside the same boundary. We cover what that looks like in enterprise deployment.

What you actually give up, and what you get

The honest summary: you trade an afternoon for a project, and you get control of the four things above. If none of the four thresholds applies to you, do not buy anything. The Custom GPT is genuinely the right answer for a folder.

The question nobody asks in the demo

Whatever you end up buying, ask the vendor to make it fail.

Give it a question your library genuinely does not cover — not a trick question, just an ordinary one about a neighbouring topic. A system with an honest failure mode will tell you it found nothing and suggest what would change the outcome. A system without one will produce a fluent, plausible, unsourced paragraph, and you will have learned the most important thing about it in fifteen seconds.

Then ask it something your library covers in exactly one place, and click the citation. If you land on the page with the passage marked, the citation is doing verification. If you land on a file, it is doing reassurance.

Those two questions separate the field faster than any feature matrix, including the one above. How to turn them into a repeatable test rather than a demo trick is the subject of how to evaluate a RAG chatbot before you buy it.

Frequently asked questions