Good AI vs. Bad AI: Our Takeaways from the Haverhill AI Summit

On June 18, the Haverhill AI Summit brought local business owners, manufacturers, students, and city leaders to Pinnacle at Harbor Place. Joe Khoury and I gave a session called Good AI vs. Bad AI: A Buyer's Field Guide for Small Businesses.

We kept it practical. Most people in the room had already tried an AI tool and been underwhelmed, so we didn't pitch. We gave them a field guide they could use the following Monday. Here is the short version.

The tool isn't the hard part

The failure numbers are real. RAND's research found that more than 80% of AI projects fail, about twice the rate of IT projects that don't involve AI. When you look at why, the cause is rarely the model. It is usually that nobody defined the problem, nobody owned the project, or nobody measured anything.

That is good news, because those failures are avoidable. They are management problems, and a small business can fix them faster than a big company can.

To the manufacturers in the room: you are not behind. Most small manufacturers haven't started yet. But the gap between you and the large players is growing, and right now is the cheapest time to close it.

What good AI looks like, and what bad AI looks like

Good AI:

• Targets one painful, frequent job
• Has a named owner
• Is measured within 30 days
• Helps your people do more instead of replacing them
• Keeps your data yours

Bad AI:

• Starts with "let's do AI" instead of a problem
• Has no owner and no metric
• Tries to replace people first
• Locks you into a vendor
• Is automation theater: impressive demos that don't change how work gets done

One of our manufacturing clients rewrote a quoting engine with AI help. The result matched the old system to within a few dollars, landed weeks ahead of schedule, and ran in parallel with the old system before cutover, so nothing was at risk. That project followed every rule in the "good" column: one painful workflow, one owner, a clear measure of success.

Before you spend a dollar

• Pick one workflow that is frequent and painful.
• Write down how it works today. Writing it down is often half the value.
• Name an owner. Use a person's name, not a department.
• Define success in 30 days: hours saved, errors avoided, or days shaved off.
• Augment before you automate. Let AI draft and assist first. Hand it more only after it earns it.

Joe added a simple way to find your first workflow: keep a two-day work log. Write down every task as you do it. The repetitive, low-judgment items jump off the page.

Six questions to ask any AI vendor

We told the room to photograph this slide:

• Will our data train your models? This question separates serious vendors from the rest. Ask it first.
• What specific job does this do, and how will we measure it in 30 days?
• Where does our data live, and who can see it?
• Does it read and write our actual systems, or just sit next to them?
• What is the real total cost, including setup, training, and upkeep?
• Can we leave, and take our data and prompts with us?

What should it cost?

Start small. A first month can cost little or nothing beyond the tool subscriptions you may already pay for. Prove the value, then expand. A simple way to set the budget: the hours it saves, times what those hours cost you. If you can't estimate the hours, you're not ready to buy yet.

Trust is built, not bought

• Keep a human in the loop on anything that matters.
• Scope access so the AI can reach only what the job needs.
• Keep an audit trail of what it did and why.
• Keep client data out of public consumer tools.

The line from the talk that stuck: "If you can't see what it did and why, you can't trust it."

Three takeaways

• One job, one owner, one metric.
• Ask the hard questions before you buy.
• Augment your people. Trust is earned, not installed.

Thank you to the organizers, to everyone who came, and to Joe for taking half the stage. If you want to see where your business stands, the free AI readiness assessment takes about ten minutes.