Public reading view · SmartAItomation

LinkedIn posting schedule

47 posts in the queue. The posting plan has one slot every day at 11:30 Europe/Chisinau. Only posts you schedule receive dates; the other version of a scheduled hook is an Alternative.

This is a manual publishing queue. Copying or marking done does not publish to LinkedIn.

View-only on this device. Scheduling, Done and improvements need the private editing link opened on this phone. Unlock editing

6/47marked done

Editorial score estimates draft readiness from length, example, verification step, team lesson and question. Edit it after review. “Improved” means the draft text was revised.

Hook 3

Your AI judge says “pass.” Your support team says the answer is wrong.

Hook details

Hook 3 · Version A · Misconception Fix (original structure)

Your AI judge says “pass.” Your support team says the answer is wrong.

Score 95/100Planned
Show postHide post
Your AI judge says “pass.” Your support team says the answer is wrong. The tempting assumption is: “An automated judge gives us an objective quality score.” It gives you a decision based on its instructions and the evidence you supplied. The decision can miss the requirement that matters to a customer. Hypothetical example: an order agent says it cancelled a request, but the tool rejected the action. A rubric that checks tone and the final sentence may mark the reply as passing without checking the resulting state. Test the judge before trusting its dashboard: → Compare it with human-reviewed examples → Include known bad answers that look polished → Check tool results and resulting state, not just style → Inspect false passes by failure type Then revise the rubric and repeat the comparison on held-out cases. A higher judge score only helps if the judge catches the failures your team needs to prevent. Evaluate the answer. Evaluate the evaluator. Evals are the most important quality control for agents, but a judge is not automatically trustworthy. Engineers need coaching to label tricky cases, challenge false passes and retest the rubric. Otherwise a demo can ship with a reassuring score that misses a customer-visible failure. Comment “JUDGE” if you want the short guide to finding false passes in an LLM judge. P.S. We'll start sending it once at least 10 people ask for it here. P.P.S. Connect with me so I can send it by DM after that.
Unlock to schedule Unlock to mark done

Copy post includes bold formatting when the destination supports it. Copy with bold letters uses special Unicode characters; they may be less accessible to screen readers.

Hook 6

The agent called the right tool with the wrong customer ID.

Hook details

Hook 6 · Version A · Problem-Example-Solution (labeled example)

The agent called the right tool with the wrong customer ID.

Score 95/100Scheduled
Show postHide post
The agent called the right tool with the wrong customer ID. Tool selection is only one part of a safe agent action. The arguments and the authority to use them matter just as much. Hypothetical example: Request: “Show me the latest invoice for my account.” Tool: get_invoice — the correct function. Argument: customer_id = a different account copied from earlier context. Result: The tool returns an invoice the requester should not see. Failure: The agent supplied a plausible but unauthorized identifier. Solution: Bind the account identifier to the authenticated session on the server. Do not let the model choose it from conversational text. Check authorization before the tool runs, and record the rejected call for review. An evaluation should inspect the proposed tool, its arguments, the authorization decision and the final answer separately. A correct-looking response cannot make an invalid call acceptable. The skill is not teaching an agent every tool name. It is teaching the team where to put deterministic controls and how to test them. Server-side authorization stops an invalid call; evals are the most important way to keep checking that boundary. Engineers need training to write adversarial cases, inspect tool arguments and count a safe rejection as success. Those skills transfer to every later agent with account-scoped actions. Comment “ACCESS” if you want the short guide to testing agent tool authorization. P.S. We'll start sending it once at least 10 people ask for it here. P.P.S. Connect with me so I can send it by DM after that.
Unlock to schedule Unlock to mark done

Copy post includes bold formatting when the destination supports it. Copy with bold letters uses special Unicode characters; they may be less accessible to screen readers.

Hook 11

CTOs: how many AI failures can your current team reproduce?

Hook details

Hook 11 · Version A · Client question answered

CTOs: how many AI failures can your current team reproduce?

Score 100/100Done
Show postHide post
CTOs: how many AI failures can your current team reproduce? The useful answer is not a large number. It is whether an engineer can take a reported failure and run it again with enough context to find the cause. Suppose a field engineer says a RAG assistant gave the wrong pressure limit. Can the team recover the question, the recorded unit, the table it retrieved, the model input, the answer and the expected behavior? Without that trail, a prompt edit may improve the screenshot while leaving the actual cause untouched. Start with a small sample of failures your team already knows. For each one, ask: → Can we reproduce it? → Can we tell which stage failed? → Can we define what a correct result would be? → Can we rerun it after a change? If the answer is “no” at any step, that is a capability to build before debating which model to buy or whom to hire. For a CTO, evals are the most important AI capability because they turn vague failures into engineering decisions. Coach the existing team to reproduce, label and investigate cases; that shared method can support a second and third product, not only rescue the first demo. Our coaching focuses on helping existing engineers practise this process across projects. Comment “TRACES” if you want the short guide to reproducing AI failures from traces. P.S. We'll start sending it once at least 10 people ask for it here. P.P.S. Connect with me so I can send it by DM after that.
Unlock to schedule Unlock to mark done

Copy post includes bold formatting when the destination supports it. Copy with bold letters uses special Unicode characters; they may be less accessible to screen readers.

Hook 27

I teach teams to test the AI judge before trusting its score.

Hook details

Hook 27 · Version A · Misconception Fix (original structure)

I teach teams to test the AI judge before trusting its score.

Score 95/100Done
Show postHide post
I teach teams to test the AI judge before trusting its score. The assumption I challenge is: “Once the judge agrees with us on a few examples, we can automate the rest.” Agreement on familiar examples is a start. It does not show whether the judge catches the failures your users care about. Hypothetical RAG example: an answer reads “80” from a pressure table but omits whether the value is Celsius or Fahrenheit. A judge trained to reward citations and completeness may pass a precise-looking number when the required unit is unresolved. Before using the score to make a release decision: → Have a domain expert label acceptable and failing answers → Record a short critique for each decision → Keep prompt examples separate from held-out validation → Count false passes and false alarms by failure type If the judge misses the ambiguous-unit case, revise the rubric and rerun unseen examples. If it still misses critical failures, keep human review where it matters. Judge the answer. Challenge the judge. Evals are the most important discipline because the judge itself needs evaluation. Engineers need practice writing rubrics, labeling persuasive failures and reviewing false passes. A judge that misses problems before users do can make a demo look ready. Comment “RUBRIC” if you want the short guide to validating an evaluation rubric. P.S. We'll start sending it once at least 10 people ask for it here. P.P.S. Connect with me so I can send it by DM after that.
Unlock to schedule Unlock to mark done

Copy post includes bold formatting when the destination supports it. Copy with bold letters uses special Unicode characters; they may be less accessible to screen readers.

Hook 38

We reviewed 10 synthetic RAG-agent cases. Six ended without an answer.

Hook details

Hook 38 · Version A · Problem-Example-Solution (labeled example)

We reviewed 10 synthetic RAG-agent cases. Six ended without an answer.

Score 100/100Done
Show postHide post
We reviewed 10 synthetic RAG-agent cases. Six ended without an answer. In our saved development lab, a hidden search parameter enum produced 13 invalid-kind errors across those explanation cases. A better final-answer prompt was not the first fix to test. Illustrative trace from that observed failure category: Request: Explain a technical requirement using the current source. Tool call: Search with a document kind the tool does not accept. Tool result: Validation error. Agent result: Repeats the bad argument or uses up its turns without a final answer. Failure: The tool contract hid allowed values and the loop did not recover cleanly. Solution: Publish valid kinds in the tool schema, return allowed values with validation errors, and reserve a final-response path when tool calls fail. Then evaluate the whole task again. Did the user get a supported answer or a useful statement of what is missing? Inspect the first failing tool call separately from the final response so the team can tell whether the repair addressed the cause. These are synthetic development cases, not a production failure rate. They show why agent evals need the tool trace as well as the answer. For agents, evals are the most important route from an empty reply to a specific fix. Teaching engineers to inspect the first tool failure, define an acceptable fallback and rerun the case gives them a diagnostic method they can reuse on another agent. Comment “TOOLS” if you want the short guide to recovering from rejected agent tool calls. P.S. We'll start sending it once at least 10 people ask for it here. P.P.S. Connect with me so I can send it by DM after that.
Unlock to schedule Unlock to mark done

Copy post includes bold formatting when the destination supports it. Copy with bold letters uses special Unicode characters; they may be less accessible to screen readers.

Hook 2

Another sprint of prompt changes. Can anyone show what improved?

Hook details

Hook 2 · Version A · Misconception Fix (original structure)

Another sprint of prompt changes. Can anyone show what improved?

Score 100/100Done
Show postHide post
Another sprint of prompt changes. Can anyone show what improved? One belief keeps this loop going: “The new answer looks better, so the prompt is better.” That may be true for the example on your screen. It does not tell you what changed across the work your users actually send. Imagine an email agent that now explains refund timing correctly. The revised prompt might still ignore the cancellation request in the same message or claim a refund was started when no tool action succeeded. Before another rewrite, keep a small fixed set of examples: → Common requests → Missing or ambiguous information → Failures your team has already seen → Cases the previous version handled well Define what passing means for each. Compare request coverage, tool outcome and final answer on the same cases. Inspect improvements and regressions separately. A final answer can look better while the first failing step remains upstream. Change the prompt. Measure the behavior. Evals are the most important feedback system for a production AI product. But choosing representative cases and deciding what counts as improvement are learned engineering skills. Coach the team through that loop once; it can use it again on the next product. Comment “PROMPTS” if you want the short guide to comparing prompt changes against a fixed eval set. P.S. We'll start sending it once at least 10 people ask for it here. P.P.S. Connect with me so I can send it by DM after that.
Unlock to schedule Unlock to mark done

Copy post includes bold formatting when the destination supports it. Copy with bold letters uses special Unicode characters; they may be less accessible to screen readers.

Hook 40

We helped a UK startup team improve its reported RAG accuracy from 65% to 96% after a two-day workshop.

Hook details

Hook 40 · Version A · Problem-Example-Solution (scoped client story)

We helped a UK startup team improve its reported RAG accuracy from 65% to 96% after a two-day workshop.

Score 90/100Scheduled
Show postHide post
We helped a UK startup team improve its reported RAG accuracy from 65% to 96% after a two-day workshop. The startup had spent a year developing an email-answering system for an oilfield production company. This is one owner-supplied engagement result, not a typical outcome or a guarantee. The available case notes do not define the metric or establish which technical change caused the improvement. The transferable lesson is a check your team can run before its next RAG change: Input: A representative question and the source material needed to answer it. Failure: The answer is wrong or unsupported. Diagnosis: Did retrieval find the right source? Did the answer use it faithfully? Did it cover the user's full request? Verification: Compare a focused change against the same labeled cases and inspect regressions. Keep retrieval coverage separate from answer quality. When an LLM judge is involved, test its decisions against expert-reviewed failures before using its score as evidence. The aim of coaching is a method your engineers can repeat across projects, not a promise to reproduce one client's number. For production RAG, evals are the most important transferable capability. Coaching engineers to define cases, inspect retrieval and test the judge gives them a method for the next product. One reported client improvement is useful context, not a guarantee that another team will reach it. Comment “RAG” if you want the short guide to separating retrieval from answer quality. P.S. We'll start sending it once at least 10 people ask for it here. P.P.S. Connect with me so I can send it by DM after that.
Unlock to schedule Unlock to mark done

Copy post includes bold formatting when the destination supports it. Copy with bold letters uses special Unicode characters; they may be less accessible to screen readers.

Hook 33

Our judge got 48 of 48 held-out synthetic decisions right. Here’s why that still doesn’t establish production accuracy.

Hook details

Hook 33 · Version A · Failure teardown

Our judge got 48 of 48 held-out synthetic decisions right. Here’s why that still doesn’t establish production accuracy.

Score 100/100Done
Show postHide post
Our judge got 48 of 48 held-out synthetic decisions right. Here’s why that still doesn’t establish production accuracy. In a controlled RAG workshop exercise, a score threshold was selected on 32 authored variants. The frozen threshold then classified 48 held-out authored variants correctly. That is a useful check of this exercise. It is not independent human validation and it does not show how the judge handles live customer traffic. The first question is what the variants cover. Did they include outdated sources, missing evidence, wrong units, partial answers and confident claims that the retrieved context never supported? The second is whether domain experts would agree with the labels. Keep the stages separate: → Prompt examples for instruction → Development cases for rubric and threshold changes → Untouched cases for final comparison → Sampled production traces for new failures Report false passes on critical slices, even if the overall held-out score is perfect. When the domain, model or traffic changes, review the judge again. A controlled success tells you the evaluator passed its test. It does not tell you the test represents tomorrow's work. Evals remain the most important production safeguard because a perfect synthetic score can hide an untested failure class. Coach engineers to expand coverage from real incidents and recalibrate the judge. The repeatable process matters more than one good demo score. Comment “HOLDOUT” if you want the short guide to keeping a synthetic judge score honest. P.S. We'll start sending it once at least 10 people ask for it here. P.P.S. Connect with me so I can send it by DM after that.
Unlock to schedule Unlock to mark done

Copy post includes bold formatting when the destination supports it. Copy with bold letters uses special Unicode characters; they may be less accessible to screen readers.

Hook 4

The assistant cited a policy. It was last year’s policy.

Hook details

Hook 4 · Version A · Failure teardown

The assistant cited a policy. It was last year’s policy.

Score 100/100Planned
Show postHide post
The assistant cited a policy. It was last year’s policy. The first reaction is often to tell the model to “use current documents.” That instruction cannot fix a retrieval path that still serves an obsolete file. In a hypothetical company-search system, a customer asks about returns. The answer quotes a 30-day rule and shows a citation. The current policy says 14 days. The citation makes the error look credible. Trace the failure in order: 1. Is the current policy present in the source collection? 2. Are version dates and document status available to retrieval? 3. Which passages were actually selected for this question? 4. If both versions were supplied, why did the answer rely on the old one? The fix depends on the stage. Update the source or filters if retrieval selected old evidence. Investigate answer instructions if the current passage was supplied but ignored. Save the question and expected answer as a regression case. Rerun it when documents, retrieval or prompts change. A citation proves where text came from. It does not prove the source was current. Evals are the most important production check here because a citation can look credible while the policy is obsolete. Coach engineers to test source authority, retrieval and answer use separately. That diagnostic flow can be reused when the next document-backed product launches. Comment “SOURCES” if you want the short guide to testing document freshness in RAG. P.S. We'll start sending it once at least 10 people ask for it here. P.P.S. Connect with me so I can send it by DM after that.
Unlock to schedule Unlock to mark done

Copy post includes bold formatting when the destination supports it. Copy with bold letters uses special Unicode characters; they may be less accessible to screen readers.

Hook 18

Data leads: your test set is large. How many cases reflect actual user failures?

Hook details

Hook 18 · Version A · Misconception Fix (original structure)

Data leads: your test set is large. How many cases reflect actual user failures?

Score 100/100Planned
Show postHide post
Data leads: your test set is large. How many cases reflect actual user failures? One assumption deserves a check: “More test examples mean better coverage.” More examples help only if they represent the behaviors you need to judge. A thousand easy requests can still miss the one failure that blocks release. Imagine a RAG test set filled with direct FAQ questions. Production brings follow-ups that rely on earlier context, obsolete documents, access restrictions and requests the corpus cannot answer. A system can score well on the large set while citing the wrong source for the cases users struggle with. Inspect the composition, not just the count: → Which failures came from real user or reviewer reports? → Which inputs are missing, ambiguous or noisy? → Which requirements are critical even if rare? → Which cases are reserved for final verification? Label the expected behavior and review results by failure type. Add cases deliberately; do not let duplicates inflate confidence. Count the examples. Cover the failures. The most important part of an eval is whether it covers failures users encounter. Engineers need coaching to turn incidents into safe, representative cases, label acceptable outcomes and maintain difficult slices. A large dataset alone will not teach the team to ship the next AI product. Comment “DATASET” if you want the short guide to covering real failure slices in an eval set. P.S. We'll start sending it once at least 10 people ask for it here. P.P.S. Connect with me so I can send it by DM after that.
Unlock to schedule Unlock to mark done

Copy post includes bold formatting when the destination supports it. Copy with bold letters uses special Unicode characters; they may be less accessible to screen readers.

Hook 39

We examined 40 synthetic RAG-agent cases. Eleven failed the whole task.

Hook details

Hook 39 · Version A · Failure teardown

We examined 40 synthetic RAG-agent cases. Eleven failed the whole task.

Score 85/100Planned
Show postHide post
We examined 40 synthetic RAG-agent cases. Eleven failed the whole task. The agent completed 29 of 40 preselected development cases in our lab. That number tells us which whole tasks passed. It does not tell us what to fix next. For the failed cases, inspect the first point where the run departed from the expected path. Some hit invalid document-kind arguments. Others had the right action idea but added an unsupported field, confused the output field for a quantity, or failed to preserve a constraint from the question. Start with end-to-end success. Then open the trace: → User request and relevant history → Tool choice and arguments → Tool result and retrieved evidence → Final claim and resulting state Mark the first upstream failure before cataloguing its downstream symptoms. A wrong final sentence may be the result of a missing source, not the original defect. These 40 were synthetic development cases. Routing, filters and tool behavior changed together, so their result does not isolate one component or predict production accuracy. Whole-task evals are the most important scoreboard; trace checks show the first repair to attempt. Coach engineers to build a failure taxonomy and check each proposed fix. That repeatable flow helps a second agent reach production instead of becoming another polished demo. Comment “AGENT” if you want the short guide to diagnosing whole-task agent failures. P.S. We'll start sending it once at least 10 people ask for it here. P.P.S. Connect with me so I can send it by DM after that.
Unlock to schedule Unlock to mark done

Copy post includes bold formatting when the destination supports it. Copy with bold letters uses special Unicode characters; they may be less accessible to screen readers.