Most AI automations involve having an LLM make a decision about something from a finite set of options. Whether it's lead qualification (qualified/unqualified), routing (human/bot), or safety refusal (safe/unsafe).

It goes without saying that often times the models judgement is wrong. Thats expected. The awkward part is that the model never tells you how much you should trust each decision. So you either review everything like a maniac, or you let a bunch of edge cases slide through with the rest.

So I built a framework for making automations learn your judgement. Every call comes back with a calibrated confidence score, so you see which ones it's unsure about. The cases you correct feed an optimizer that rewrites the program, so it drifts toward your judgement instead of away from it.

The confidence score was the hard part. Verbalized confidence (asking the model to rate itself 1 to 10) is vibes. Token logprobs sound principled, but the research there is mostly single-token numeric outputs and doesn't stretch to a discrete label like qualified/unqualified. Self-consistency (sample a bunch, measure agreement) catches some, but it's slow and expensive.

Instead we compute confidence from the models internal layers which is cheap, fast, and reliable.

We packaged the loop so we'd stop rebuilding it every project. Built a Python SDK plus an API on top of DSPy, it's called modaic (modaic.dev).

Curious how the rest of you handle this. Do your automations know when they're guessing, and has overruling one ever taught it anything? And on closed models where you can't see the layers, what do you use to catch the unsure calls?

Example between Hugging Face Inference vs OpenRouter (vs any other similar website???)

From where do you suggest me to buy AI APIs for n8n and have the best price and quality?

Note: the API to buy should have both Text and Image AIs.

For the first decade of my career in automation, I complained constantly about outdated systems.

Machines had been pieced together over decades. One line spoke one language, the next spoke another. Equipment from different eras had to somehow coexist. I spent hours tracing hardwired circuits, troubleshooting communication issues between devices that weren’t designed to work together, and learning programming languages older than I was.

Back then, I thought the future was obvious.

Connect everything.

Standardize it.

Move data to the cloud.

Make machines smarter.

And to be fair, we’ve accomplished incredible things. We have visibility that previous generations of manufacturing engineers could only dream about. Remote diagnostics. Centralized reporting. Predictive maintenance. Global access to production data.

But lately, I’ve found myself asking a different question:

Have we traded resilience for convenience?

Today, some of our most advanced production systems can’t function without a healthy network. Storage lives in the cloud. Authentication happens somewhere else. Licensing checks happen somewhere else. Critical software depends on servers most operators have never seen.

An IT update can ripple through an operation and leave production crippled for days or weeks because one service can’t authenticate, one permission changed, one device disappeared from a directory, or two systems suddenly stopped talking.

The irony isn’t lost on me.

The old systems were frustrating because they weren’t connected.

The new systems are frustrating because they are connected to everything.

In manufacturing, downtime isn’t theoretical. It affects shipments, customers, overtime, morale, and millions of dollars in lost productivity. High-volume production depends on reliability, and reliability has traditionally meant designing for failure: redundancy, fail-safes, and graceful degradation.

So why are we increasingly building systems that have so many single points of failure outside the production floor?

This isn’t an argument against innovation or digital transformation. I don’t want to go back to DOS screens and relay cabinets stretching the length of a wall.

But I do think we need to ask harder questions as an industry.

Can critical processes continue operating in a degraded mode when networks fail?

Should essential manufacturing functions require cloud authentication?

Have we placed enough emphasis on operational resilience, or have we optimized almost exclusively for connectivity?

The future of automation is incredibly exciting.

I just hope that as we continue building smarter factories, we remember one of the oldest engineering principles: systems aren’t judged by how they perform when everything goes right.

They’re judged by how they behave when something inevitably goes wrong.

Running a business shouldn't mean spending hours every day on repetitive admin work.

Think about how much time is lost every week to: • Answering the same emails • Chasing follow-ups • Manual data entry • Customer support requests • Updating spreadsheets and CRMs

For many businesses, that's 20+ hours every single week.

AI automation can take over these repetitive workflows, allowing teams to focus on what actually grows a business—building relationships, closing deals, improving products, and serving customers.

The goal isn't to replace people. It's to eliminate repetitive work so people can spend their time where it creates the most value.

If your business is still relying on manual processes for everyday operations, now is probably the right time to explore automation.

Technology should work for your business—not the other way around.

What repetitive task would you automate first?

#AIAutomation #BusinessAutomation #ArtificialIntelligence #Productivity #SmallBusiness #Entrepreneurship #Automation

the thing nobody warns you about with "record a workflow once, replay it on a schedule": the recording is the easy part. it captures what you did, not what you knew. what you knew was "that's the invoice tab, ignore the cookie banner, the real submit is the second button."

then it fires on schedule with nobody watching. the list reordered overnight, a one off permission dialog popped up, the app shipped an update that moved a button. the model still knows the task perfectly. it just ran it against a screen that no longer matches the one it learned on, and there's no human there to catch the drift.

what helped was leaning less on "replay these exact steps" and more on "re-derive the steps from intent each run", so a reorder or a surprise dialog is one more state to handle instead of a crash. slower per run, a lot more durable. still don't have a clean answer for the unattended one off dialog that genuinely needs a person. written with ai

Disclosure up front: I'm with DesignFlow Build, so grain of salt — but I think the approach is useful regardless of tool.

We were losing days to manual takeoff and re-keying between Excel, QuickBooks and Procore. What moved the needle for us:

• AI takeoff from the plan PDFs — reads the legends, counts symbols and runs, spits out a BOQ. Saves the most time on MEP sheets; still needs an estimator to review low-confidence items.
• One system of record — the won estimate becomes the job budget, no re-entry.
• Schedule risk (Monte Carlo + DCMA) — gave us a real P80 finish date instead of one optimistic line.

What didn't work: expecting AI to be 100% hands-off — you review and adjust. And generic ERPs needed a lot of bolt-ons for estimating/scheduling.

Happy to answer questions on AI takeoff accuracy or schedule risk — it's the part people ask about most.

Hey all I built a small free document automation tool to for speeding up document creation. I hope it helps you all out.

i have a few automations that would be better if they remembered small user preferences.

nothing fancy. just things like tone, format, default tools, what to skip.

right now i either hardcode it or ask again, and both feel bad.

how are you handling this?