Chris Neuwirth, VP of Cyber Risk, NetWorks Group

The question I get most often about AI is a practical one. Clients, peers and people on our own team are interested in hearing what I actually use it for on a Tuesday afternoon. There is plenty written about what attackers are doing with these models and what vendors promise they will do for defenders. Far less is written by practitioners about where the models sit in our daily work, where they help and where we keep them away from the decisions. This post aims to paint a picture of how I work today, and how our broader Ethical Hacking team weaves AI into their engagements.

The Platform Comes First

As we started weaving AI into our work, the most important choice we made had nothing to do with which model to use. Client data is the most sensitive thing we handle, so before AI touched any engagement, I built the environment it works inside. It runs on infrastructure we control, and engagement data, recordings and evidence stay there by default. Commercial cloud models handle harder reasoning and drafting work, under an internal AI governance policy that sets which tools can handle which types of data. Material the platform retrieves for a cloud model is labeled by sensitivity and logged, so we can review how it was used. 

We use two broad categories of models, connected to our work through custom software. Frontier models are the largest and most capable models from the major AI labs, offered as cloud services. They handle hard reasoning. We use them through paid accounts and APIs with training on our data turned off. Open-weight models are models whose underlying files are published, so we can download them and run them on our own hardware. Those are what we run on our own infrastructure, and they are our default for sensitive client work. Custom software we built around the models (what the industry calls a harness) connects them to our tools, evidence and methodology, and routes each task to a model. The harnesses are where we spent most of our time engineering custom solutions, and they stay the same when we replace a model.

Another key element to consider is memory. A chat history alone is not enough to serve as managed company memory. Our platform keeps a governed, searchable record of methodology, prior findings, meeting notes and decisions, and every entry in it can be audited and corrected. When I start a new engagement, the platform can retrieve our finding-writing standards, relevant severity precedents and that client's kickoff notes for the model to use. The models themselves are interchangeable; we have swapped them several times this year without changing how anyone works. We focused our time and effort on the harnesses, the memory and the guardrails.

Penetration Testing

On an engagement, AI shows up in three places.

Preparation. Our team gets a constant stream of offensive security research: new Active Directory certificate abuse paths, relay techniques, cloud identity attacks and more. We use models to distill that research into a field manual organized by phase, with the exact commands, success criteria, detection notes and source for each technique. Before an internal test, we make sure the relevant playbooks are current, and when a tester hits an unfamiliar configuration mid-engagement, finding the answer requires a simple search rather than an afternoon of reading.

Analysis. Some work simply takes more reading than a test window allows. Firmware for an unfamiliar chip architecture is a good example. With a model assisting the reverse engineering, a tester can map the image, locate the update logic and flag where authenticity checks look weak far sooner than by reading alone. That analysis gives the tester specific issues to validate on real hardware before treating the device behavior as confirmed. Attack-path graphs from BloodHound, large configuration exports and application source get the same treatment: the model helps analyze the engagement material provided; the tester validates the conclusions.

Reporting. Reports are where AI saves the most time and where we are strictest. Models draft the attack narrative from the evidence we collected, compare a proposed severity rating with comparable, previously validated findings, and draft recommended remediation steps a client's team can follow to address misconfigurations and other findings. The practitioner determines the final severity from the evidence and the client's context. Every claim must be traceable to evidence (screenshots, command outputs, captured requests, etc.). If a sentence cannot be traced, it comes out. We also have a standing rule that nothing the model proposes gets executed against a client environment if it could destabilize a production service.

Assessments and Interviews

Organizational assessments involve a lot of listening. We conduct interviews, run surveys and collect documents, and the analysis has to reconcile all of it. Models turn the recordings into transcripts, group survey responses by theme and cross-reference what one person said in an interview against what another wrote in a survey. That work now takes a fraction of the time it used to.

That speed made verification more important. A model may produce a confident summary that drops a qualifier from a quote or reports that nobody else raised a concern when three people did. Our standard now is that every statement in an assessment finding must be traceable to a primary source identified in our internal evidence record. That traceability does not require naming interviewees in the report. Quotations preserve the speaker’s meaning and relevant qualifiers, and a claim like "only one person mentioned this" gets checked against the full record before it is written down. 

The Rules We Kept

A few principles hold across all of this work:

• Client data stays on infrastructure we control by default. Our AI governance policy sets which tools can handle which types of data.
• Nothing is sent to a client or other recipient without human approval. The system drafts emails, reports and documents. It does not send them.
• The practitioner owns the result. A model can find, draft and summarize. The tester defends the finding, and the name on the report belongs to a person.
• Verify everything. Fluent output is not evidence.

Questions for Your Own Program

If your organization is weighing how to use AI in security work, the same questions apply:

• Do you know which models your data reaches, and where they run?
• Can you audit and correct what your AI tools retain between sessions?
• Which actions can an AI tool take on its own, and which require a person to approve?
• When an AI-assisted result is wrong, who is accountable, and how would you find out?

The models will keep changing. The answers to those questions determine whether AI makes your security work better or simply makes it faster.

‍

Published By: Chris Neuwirth, VP of Cyber Risk, NetWorks Group

Publish Date: October 23, 2026