When to Avoid or Constrain LLM Use

Why “avoid or constrain” is a first-class design choice

Using an LLM is not an all-or-nothing decision. Many real systems work best when the model is either not used at all for certain tasks, or used only inside strict boundaries (limited inputs, limited outputs, limited actions, and mandatory checks). This chapter focuses on identifying situations where the safest, cheapest, or most reliable option is to avoid LLMs entirely, and on patterns for constraining them when you still want their benefits.

“Avoid” means you choose deterministic software, a database query, a rules engine, a traditional ML model, or a human workflow instead of an LLM. “Constrain” means the LLM can help, but only in a controlled role: drafting, summarizing, classifying, or proposing options—while other components enforce truth, policy, and safety.

Situations where you should avoid LLM use

1) When the task requires guaranteed correctness (not “usually right”)

If the output must be correct every time, an LLM is rarely the right core engine. Examples include tax calculations, dosage computations, cryptographic operations, financial ledger updates, and compliance-critical reporting where a single error is unacceptable.

• Better approach: deterministic code, audited libraries, and validation rules.
• LLM role (if any): explain results in plain language after the deterministic system produces the authoritative answer.

Practical example: Instead of asking an LLM to compute payroll deductions, compute them in code and ask the LLM to generate an employee-friendly explanation of the line items, using the computed numbers as fixed inputs.

2) When you need stable, repeatable outputs across time

Some workflows require the same input to produce the same output for auditability or fairness (for example, eligibility decisions, standardized scoring, or regulated disclosures). LLM outputs can vary due to model updates, sampling settings, or subtle prompt differences.

Continue in our app.

You can listen to the audiobook with the screen off, receive a free certificate for this course, and also have access to 5,000 other free online courses.

Or continue reading below...

Download the app

• Better approach: rules, templates, or a versioned model with strict decoding settings plus regression tests.
• LLM role (if any): generate drafts that are then normalized into a fixed template by deterministic logic.

3) When the task is already solved by a simple, cheaper method

Many “LLM-shaped” problems are actually string matching, routing, or CRUD operations. If a keyword search, a dropdown, or a small set of rules solves the problem, an LLM adds cost and failure modes.

• Better approach: search indexes, FAQ routing, decision trees, or form-based data collection.
• LLM role (if any): only for free-form user input normalization (e.g., mapping messy text into a known category), with strict validation.

4) When you cannot control data exposure or retention requirements

If you are handling sensitive data (personal identifiers, health information, confidential contracts, trade secrets) and you cannot ensure appropriate controls (data minimization, access control, retention policies, vendor terms, and logging discipline), avoid sending that data to an LLM service.

• Better approach: keep processing on systems with established security controls; use redaction; use on-prem or isolated deployments if required.
• LLM role (if any): operate on de-identified or synthetic data; summarize locally; or use a constrained extraction pipeline that never transmits raw sensitive text.

5) When the model would be asked to make high-stakes decisions about people

Hiring, lending, insurance, medical triage, legal judgments, and disciplinary actions are high-stakes. Even if an LLM is “only advisory,” it can strongly influence outcomes. Risks include hidden bias, inconsistent reasoning, and lack of traceability.

• Better approach: transparent criteria, human review, and auditable decision logic.
• LLM role (if any): assist with administrative tasks (summarizing documents, drafting communications) while explicitly prohibiting it from recommending outcomes.

6) When you need authoritative citations and traceable provenance

If you must provide verifiable sources for each claim (e.g., regulatory submissions, academic writing, legal briefs), an LLM alone is not sufficient. You need a pipeline that ties statements to specific documents and passages.

• Better approach: retrieval and citation tooling, document management systems, and human verification.
• LLM role (if any): rewrite or summarize only from provided excerpts, and require quoted evidence for every claim.

7) When the output could create security vulnerabilities

Asking an LLM to generate code, infrastructure configurations, or security policies can introduce subtle vulnerabilities (injection risks, insecure defaults, dependency issues). If you cannot review and test thoroughly, avoid using an LLM for production-critical code or configs.

• Better approach: vetted templates, security-reviewed libraries, static analysis, and peer review.
• LLM role (if any): propose options, explain tradeoffs, or generate test cases, but never merge directly to production without review.

8) When the system could be manipulated by untrusted users

If users can supply arbitrary text that influences what the LLM does (support chat, agentic workflows, document ingestion), you must assume adversarial inputs. Without strong constraints, the model can be tricked into leaking data, ignoring policies, or taking unintended actions.

• Better approach: avoid LLM-driven actions; use strict input validation and sandboxing; keep the model away from secrets and privileged tools.
• LLM role (if any): limited to non-sensitive tasks like tone polishing, with no access to internal systems.

Situations where you should constrain LLM use (not avoid it)

1) Drafting and rewriting with fixed facts

LLMs are strong at turning bullet points into readable text. Constrain them by providing the facts as structured inputs and forbidding new facts.

• Constraint: “Use only the provided data; if something is missing, ask a question.”
• Enforcement: validate that all numbers and named entities appear in the input; reject outputs that introduce new entities.

2) Summarization with bounded scope

Summaries can drift. Constrain by limiting the source material and requiring extractive anchors (quotes or references to sections).

• Constraint: “Summarize only what is in this document; include 3 direct quotes that support the summary.”
• Enforcement: check that quotes exist verbatim in the source.

3) Classification and routing

Using an LLM to map free-form text into a small set of categories is often effective, but you should constrain the output to a closed set.

• Constraint: output must be one of: {Billing, Technical Issue, Account Access, Cancellation, Other}.
• Enforcement: schema validation; fallback to “Other” if uncertain; log for review.

4) Brainstorming and option generation

For ideation, the cost of being “wrong” is low, but you still want guardrails: avoid unsafe suggestions, ensure relevance, and prevent confidential leakage.

• Constraint: “Provide 10 options; do not include customer data; do not propose actions requiring admin access.”
• Enforcement: content filters and redaction; keep the model isolated from sensitive sources.

A practical decision checklist: avoid vs constrain

Use the following checklist before adding an LLM to a workflow. If you answer “yes” to any avoid item, strongly consider not using an LLM for that step. If you answer “yes” to constrain items, you can still use an LLM but only with guardrails.

Avoid checklist (any “yes” is a red flag)

• Does this step require mathematically exact results or zero-defect correctness?
• Would a single mistake cause legal, financial, medical, or safety harm?
• Is the output required to be identical for the same input (audit/fairness)?
• Are you unable to review outputs before they take effect?
• Does the step require handling sensitive data you cannot safely transmit or log?
• Could an untrusted user influence the model to access secrets or take actions?

Constrain checklist (if “yes,” add guardrails)

• Can you provide the authoritative facts as inputs and restrict the model to them?
• Can you validate outputs with code (schema, allowed values, numeric checks)?
• Can you require evidence (quotes, references) for claims?
• Can you keep the model from taking direct actions (approve, pay, delete, send)?
• Can you add human review for high-impact outputs?

Constraint patterns you can implement

Pattern 1: Separate “language” from “decision”

Let deterministic logic decide; let the LLM communicate. This is one of the most reliable ways to use LLMs safely.

• Example: A refund policy engine determines eligibility and amount. The LLM writes a polite email explaining the decision and next steps, using only the policy engine’s outputs.

// Pseudocode: decision first, language second
eligibility = policy_engine.check_refund(order)
message = llm.generate_email({
  "decision": eligibility.status,
  "amount": eligibility.amount,
  "reasons": eligibility.reasons,
  "allowed_actions": ["offer_store_credit", "escalate_to_agent"]
})

Pattern 2: Closed-world prompting + refusal on missing info

Constrain the model to a “closed world”: it may only use the provided context. If information is missing, it must ask clarifying questions rather than guessing.

• Implementation idea: include an instruction and a required output field called missing_information.

Return JSON with keys: answer, missing_information.
Rules:
- Use only the provided context.
- If context is insufficient, set answer to null and list what is missing.

Pattern 3: Structured outputs with schema validation

Free-form text is hard to validate. Prefer structured outputs (JSON) and validate them strictly. If validation fails, retry with a corrective prompt or fall back to a safer path.

• Use cases: routing, extraction, form filling, ticket triage.
• Validation: required fields, type checks, allowed enums, max lengths.

// Example schema idea
{
  "category": "Billing|Technical|Access|Cancellation|Other",
  "priority": 1-5,
  "summary": "string (max 240 chars)",
  "needs_human": true|false
}

Pattern 4: Tool use with least privilege

If the LLM can call tools (search, database, email, ticketing), restrict what it can do. Give it read-only access where possible, narrow scopes, and require explicit confirmations for sensitive actions.

• Least privilege: separate tools for “read customer record” vs “update customer record.”
• Two-person rule: require human approval for actions like refunds, account closures, or sending external emails.

Pattern 5: Output constraints (length, tone, forbidden content)

Many failures are not about facts but about format and policy. Constrain the output to reduce risk.

• Length limits: prevent rambling and reduce chance of introducing errors.
• Tone constraints: professional, neutral, no admissions of liability.
• Forbidden content: no personal data, no internal URLs, no credentials.

Constraints:
- Max 120 words.
- Do not include names, emails, phone numbers.
- Do not mention internal systems.
- If user asks for restricted info, reply with a refusal template.

Pattern 6: Confidence is not a control—use verifiers instead

Models can sound confident even when wrong, and self-reported confidence is not a reliable safety mechanism. Constrain by adding external checks: deterministic validators, unit tests, policy rules, and cross-checking against authoritative sources.

• Example: If the LLM extracts an invoice total, verify that the sum of line items matches the total; if not, flag for review.

Pattern 7: Human-in-the-loop for high-impact steps

When mistakes are costly, add a human checkpoint. The LLM can prepare a recommendation packet, but a person approves the final action.

• Good fit: legal correspondence drafts, medical patient instructions, HR communications, security incident summaries.
• Make review efficient: highlight uncertainties, include evidence snippets, and provide a checklist for reviewers.

Step-by-step: designing a constrained LLM workflow

This is a practical sequence you can apply to most products and internal tools.

Step 1: Define the “authoritative source of truth” for each field

List every piece of information the output might contain (dates, prices, policy rules, customer status, product specs). For each, decide where it must come from: database, policy document, human input, or not allowed at all.

• Deliverable: a table mapping output fields to sources.

Step 2: Decide what the LLM is allowed to do

Write a narrow role statement. Examples: “rewrite,” “summarize,” “classify,” “extract,” “draft options.” Avoid roles like “decide,” “approve,” “diagnose,” or “authorize.”

• Deliverable: a one-sentence capability boundary.

Step 3: Choose an output format you can validate

If the result feeds another system, prefer structured output. Even for user-facing text, you can wrap it in JSON with fields like message, citations, follow_up_questions, and disclaimer_required.

• Deliverable: a schema with required fields and allowed values.

Step 4: Add deterministic guards before and after the LLM call

Before: redact sensitive fields, limit input length, and strip untrusted instructions from user-provided documents when possible. After: validate schema, scan for forbidden content, check numbers, and enforce policy rules.

• Deliverable: pre-processing and post-processing functions with tests.

Step 5: Implement safe fallbacks

Decide what happens when the model fails validation or the request is out of scope. Fallbacks might include: ask clarifying questions, route to a human agent, return a template response, or provide a search result list instead of a synthesized answer.

• Deliverable: explicit fallback paths and user messaging.

Step 6: Log for audit without oversharing

Logging is essential for debugging and safety monitoring, but logs can become a privacy risk. Log minimal necessary metadata, redact sensitive content, and set retention limits. Track validation failures and fallback rates as leading indicators of risk.

• Deliverable: a logging policy and monitoring dashboard metrics.

Common anti-patterns (and safer alternatives)

Anti-pattern: “LLM as the database”

Asking the model to recall product specs, pricing, or policy details from memory is fragile.

• Safer alternative: fetch the data from the database and have the LLM format it.

Anti-pattern: “LLM decides and executes”

Letting the model both decide what to do and perform the action (send email, issue refund, delete record) magnifies errors.

• Safer alternative: the model proposes an action plan; deterministic rules and/or a human approves; tools execute with least privilege.

Anti-pattern: “One giant prompt for everything”

Combining policy, tool instructions, user content, and formatting rules in a single long prompt makes it harder to reason about failures.

• Safer alternative: break into stages: classify intent, retrieve authoritative data, draft response, validate, then deliver.

Anti-pattern: “Relying on disclaimers instead of controls”

Disclaimers do not prevent harm if the system still produces unsafe or incorrect outputs.

• Safer alternative: enforce constraints with validation, tool restrictions, and review gates; use disclaimers only as an additional layer.

Mini case studies: applying constraints in realistic workflows

Customer support: account access issues

Risk: identity verification and account changes are sensitive. Constraint approach: the LLM can only (1) classify the issue, (2) ask for non-sensitive troubleshooting steps, and (3) generate a handoff summary for a human agent. It cannot view or modify account data.

• Step 1: LLM classifies: “Account Access.”
• Step 2: LLM provides a checklist: reset password, check email filters, verify device time.
• Step 3: If unresolved, LLM produces a structured handoff note.

Internal analytics: executive summary of KPIs

Risk: incorrect numbers or invented trends. Constraint approach: compute KPIs in SQL; pass only the computed table to the LLM; require the narrative to reference specific rows/columns; validate that every number in the narrative appears in the KPI table.

Contract review: clause extraction

Risk: missing a critical clause or mislabeling obligations. Constraint approach: use the LLM to extract candidate clauses into a structured format with quotes and page/section identifiers; require a human reviewer to confirm. The model is not allowed to provide legal advice or final determinations.

{
  "clause_type": "Termination",
  "excerpt": "...",
  "location": "Section 12.2",
  "notes": "Potential auto-renewal; confirm notice period"
}