LLM gateways, guardrail latency and quality checks in CI

Vectorize. Cloudflare provides a vectordb solution as well, with a generous free tier. Would be good to add this into my website chatbot solution.

Lighthouse. Lighthouse is an open-source, automated tool to improve the quality of web pages. It has audits for performance, accessibility and SEO. Can be integrated with CI/CD before any new release.
Lychee. Lychee link checker is a good fit for content-driven websites. Broken links can be found and fixed at the CI level itself.

Guardrails. Implementing guardrails in AI applications at times significantly increases the app’s latency.
Prompt injection. SFT used to defend LLMs against prompt-injection attacks often fails by relying on surface-level patterns rather than understanding malicious intent. Good read: arxiv.org/pdf/2601.07185
LLM gateways. Explored tensorzero while exploring open-source LLM gateway options. Another option worth exploring for enterprise: Portkey gateway. TrueFoundry, Kong are some others.
Caching. Caching, when not implemented properly, can cause data leaks. Important to consider in the AI application world.

Web scraping. Scrapling is a web scraping framework with anti-bot and Cloudflare bypass capability.
Chatbot feedback. User feedback extracted from chatbot conversations can be used for evaluation and personalization of the application.