AbbVie threat-intelligence engineer Rachel James described how her team used large language models to analyse detections, observations, correlations and rules. The work searched for similarity, duplication and defensive gaps before integrating more external threat intelligence.
OpenCTI provided a specialised platform for creating a unified threat picture. AI transformed large volumes of unstructured text into STIX, a standard format, with a longer-term goal of connecting intelligence to vulnerability management and third-party risk.
James also emphasised the OWASP Top 10 for Generative AI, prompt-injection research and red-teaming. She tracked adversary discussion and tool development through open sources and automated dark-web collections, combining defensive operations with direct study of attack techniques.
Three trade-offs were central: accepting generative unpredictability, losing transparency as models became complex and misjudging return on investment. James saw opportunity because threat-intelligence and data-science lifecycles were closely aligned, especially for shared datasets.
The development should be read in its original publication context. It described the evidence, forecasts and vendor claims available at that time; later outcomes should not be projected backwards into promises that had not yet been tested.
A useful assessment separates demonstrated functions from proposed scale. It compares the new approach with a realistic baseline, including manual work, waiting time, existing infrastructure and the cost of correcting errors.
Long-term value requires more than launch performance. Data, models, organisations and external rules change, so version records, periodic checks and a named owner are needed to detect drift and decide whether the original case still holds.
Distribution also matters. An average gain can conceal predictable weakness for a language, site, workload or user group. Breaking results down and inviting feedback from people closest to the process makes expansion decisions more defensible.
Before scaling, leaders should map the complete delivery chain. Data sources, interfaces, cloud services, suppliers and human review can each become the limiting factor, even when the most visible model performs well. That map also exposes concentration, security and recovery requirements that a product demonstration rarely shows.
Financial analysis should follow outcomes rather than activity. More generated code, processed alerts or trained participants is not automatically more value. Measures such as completed tasks, avoided incidents, verified skills, lower correction effort and improved service quality give decision-makers a stronger basis for continuing investment.
Reversibility is equally important in a fast-moving market. Portable records, documented interfaces and clear stop criteria let an organisation change provider, narrow a deployment or return a decision to people when performance, pricing or policy changes. That flexibility is practical resilience, not a lack of commitment.
LLMs Analysed Detections and Rule Gaps
Similarity analysis could reveal duplicate rules while gap analysis highlighted areas not covered by existing detections. Adding external intelligence could place internal alerts inside a wider adversary context.
The practical importance was that models needed to show the evidence behind a suggested duplicate or missing control. A credible evaluation would connect the claim to representative outcomes, record exceptions and human intervention, and identify who could act when conditions changed. It should compare different workloads or user groups and document the resources required to sustain the result, preventing one headline metric from substituting for operational evidence. Teams would also need a baseline, a review schedule and a defined threshold for pausing or changing the system. Those controls would make progress measurable while preserving room to respond when costs, risks or user needs diverged from the original assumptions.
OpenCTI and STIX Created a Shared Structure
Converting unstructured reports into STIX made information easier to connect across security tools and teams. Standardisation could link actors, techniques, vulnerabilities and observations.
The practical importance was that automated extraction required validation because a wrong relationship could misdirect investigation. A credible evaluation would connect the claim to representative outcomes, record exceptions and human intervention, and identify who could act when conditions changed. It should compare different workloads or user groups and document the resources required to sustain the result, preventing one headline metric from substituting for operational evidence. Teams would also need a baseline, a review schedule and a defined threshold for pausing or changing the system. Those controls would make progress measurable while preserving room to respond when costs, risks or user needs diverged from the original assumptions.
Adversary Research Informed Defensive Testing
James combined open-source monitoring, dark-web collection, prompt-injection development and GenAI red teaming. Understanding attacker interest helped prioritise realistic scenarios instead of hypothetical risks.
The practical importance was that defenders gained value when research moved into detections, exercises and incident playbooks. A credible evaluation would connect the claim to representative outcomes, record exceptions and human intervention, and identify who could act when conditions changed. It should compare different workloads or user groups and document the resources required to sustain the result, preventing one headline metric from substituting for operational evidence. Teams would also need a baseline, a review schedule and a defined threshold for pausing or changing the system. Those controls would make progress measurable while preserving room to respond when costs, risks or user needs diverged from the original assumptions.
AI Introduced Uncertainty and ROI Risk
Generative systems could be creative but unpredictable, opaque and easy to overvalue. Those limitations mattered in security where an unsupported conclusion could waste time or miss an attack.
The practical importance was that adoption needed measurable analyst outcomes and a clear boundary around autonomous action. A credible evaluation would connect the claim to representative outcomes, record exceptions and human intervention, and identify who could act when conditions changed. It should compare different workloads or user groups and document the resources required to sustain the result, preventing one headline metric from substituting for operational evidence. Teams would also need a baseline, a review schedule and a defined threshold for pausing or changing the system. Those controls would make progress measurable while preserving room to respond when costs, risks or user needs diverged from the original assumptions.
Across the announcement, technical capability was only one part of the system. Skills, access, integration, governance, cost and affected people could each determine whether the promised value survived outside a demonstration.
The durable lesson is to preserve traceability and choice. Organisations should document assumptions, keep consequential outputs reviewable, monitor downstream effects and retain a realistic exit route. Those controls allow ambitious AI work to improve without making early momentum an irreversible commitment.
