ClickIT Case Studies PoC: AI Knowledge Management Agent

Proof of Concept

How We Built an AI Knowledge Agent on AWS to Avoid Data Silos and Wasted Hours

Solution

AI Agent Development

Case Study

In-house

ClickIT built an AI knowledge management system, helping industries like real estate, logistics, healthcare, and manufacturing. Users can query internal documents using a chat-based interface. The system uses a serverless RAG architecture on AWS.

The solution combines Amazon Bedrock, S3 Vectors, Lambda, FastAPI, React, and Terraform to retrieve relevant information from policies, manuals, contracts, procedures, and other internal documents while grounding responses in the retrieved source material.

Key Takeaways of the PoC

Enterprise knowledge is often distributed across PDFs, Word documents, policies, technical manuals, internal procedures, and other repositories that employees have to search manually.

ClickIT designed an AI knowledge management system to make that information easier to access through natural-language questions.

The system was validated with realistic internal policy documents, but it is not yet in production use.It also has not been evaluated with formal RAG quality metrics.

The ClickIT Solution: An AI Knowledge Management System on AWS

The system follows two primary workflows:

Document Ingestion

Documents are uploaded directly to Amazon S3 through a presigned URL.

An S3 event automatically triggers ingestion, where the document is:

Knowledge Retrieval

When a user asks a question, the system:

The implementation uses 2,200-character chunks with 350-character overlap, retrieves the top 12 chunks, and rejects results below a 0.10 similarity threshold rather than forcing the model to answer from weak evidence.

How the AI Knowledge Management System Works

Secure Document Upload

Files are uploaded directly from the browser to Amazon S3 using presigned URLs.

The document body never needs to pass through Lambda, helping keep the architecture lightweight and avoiding unnecessary Lambda memory usage.

Automated Document Ingestion

Once the document reaches S3, an event triggers the ingestion Lambda.

The system parses supported documents using lightweight libraries such as:

Document chunks are converted into embeddings using Amazon Titan Text Embeddings V2.

The resulting vectors are stored in Amazon S3 Vectors, allowing the system to retrieve semantically relevant passages even when the user does not use the same terminology found in the original document.

This enables more flexible enterprise knowledge discovery than static keyword matching.

 

Grounded Answer Generation with Amazon Bedrock

 Once relevant chunks are retrieved, the system sends the context to an LLM through Amazon Bedrock.

The default generation model is Amazon Nova Lite, while Claude models are reserved for evaluation or higher-stakes validation scenarios. The model is instructed to answer only from the retrieved context and include source attribution.

When context is insufficient, the system returns an explicit no-match response instead of inventing information.

Anti-Hallucination Validation

The solution does not rely only on prompt instructions.

A code-level grounding check analyzes the generated answer and looks for numerical claims that do not exist in the retrieved source context.

If an unsupported figure appears, the answer is flagged as ungrounded.

This behavior was tested against realistic policy content to confirm that the system rejects fabricated numerical claims while accepting grounded paraphrases.

Enterprise AI Guardrails

Amazon Bedrock Guardrails provides another layer of protection around generated answers.

Configured controls include:

When the guardrail intervenes, the API returns a controlled response rather than generating unsupported source citations.

All Technologies Used

We implement a tech stack built for production-scale AI and data systems, prioritizing reliability, automation, and cloud flexibility.

AWS Logo
AWS
Amazon Bedrock Logo
Amazon Bedrock
AWS CloudFront Technology Logo
CloudFront
Logo React
React
Logo Python
Python
FastAPI
AWS Lambda
Amazon S3
Amazon S3 Vectors
Amazon Bedrock
Amazon Nova Lite

How an AI Knowledge Agent Transforms Your Daily Operations

Trusted Answers Grounded in Enterprise Knowledge

Teams can query policies, procedures, contracts, and technical documentation with answers grounded in retrieved content and a clear fallback when evidence is missing.

Technical evidence: NO_CONTEXT_MATCH behavior validated in tests/test_app.py.

Anti-Hallucination Protection Beyond Prompting

A code-level check, answer_seems_ungrounded(), validates generated responses against retrieved context to detect unsupported claims.

This adds an extra verification layer for compliance-sensitive or policy-driven use cases.

Technical evidence: Unit tests confirm fabricated figures are rejected while grounded paraphrases are accepted.

Validated Against Real Internal Documents

The RAG pipeline was tested with real, un-curated policy documents, including cases where multiple sources could apply.

This helps validate retrieval and source attribution against the complexity of real enterprise knowledge.

Technical evidence: Verified through the repository’s policy-document test fixtures.

Built-In PII and Content Protection

Amazon Bedrock Guardrails adds protection for sensitive data, prompt injection, and unsafe content.

The solution can anonymize personal information and block high-risk data such as SSNs, card numbers, and AWS credentials.

Technical evidence: Configured in infra/modules/guardrails/ and integrated through backend/app/bedrock.py.

Deployable AWS Architecture

The solution includes Terraform, FastAPI, React, deployment scripts, and CI/CD—not just an isolated AI prototype.

Teams get a reproducible foundation that can be tested, deployed, and evolved toward production.

Technical evidence: Terraform modules, deployment scripts, and .github/workflows/.

Cost-Conscious RAG Deployment

The serverless AWS architecture is estimated at approximately 5–20/month at pilot scale, depending on usage and model selection.

It also supports cost allocation by model, project, team, and environment.

Technical evidence: Cost allocation is configured in infra/cost_allocation.tf.

Estimated pilot cost, not measured production spend.

Leaner Architecture with Documented Trade-Offs

The team evaluated heavier options such as LangChain and Docling and selected lighter components better suited to serverless deployment.

This reduces unnecessary dependencies while keeping the RAG workflow easier to maintain and deploy.

Technical evidence: Architecture decisions and rationale are documented in the project’s technical proposal.

Ready to unlock your company’s hidden knowledge with AI Agents?

Trusted by Industry Leaders