Business

When the boss's email is fake: spotting AI phishing in everyday work

AI makes scam emails more convincing, but can also help with verification. This is how Austrian teams stop BEC with secure approval workflows in daily operations.

Adult employees in an Austrian office review a suspicious email on a laptop together

A message from management requests an urgent transfer, a supplier reports a new bank account, or HR asks for sensitive data. Such emails often seem credible because the tone, names and ongoing projects are correct. Generative AI makes it easier for attackers to craft error‑free, fitting texts. The same technology can help employees sort warning signals. However, it must not make the decision for them. The most effective protection remains a clear process that verifies identities via a second channel and removes time pressure from the situation.

This guide shows how Austrian businesses can use AI as a cautious analysis aid without copying confidential messages into external systems. It focuses on Business Email Compromise (BEC) and the decisions that people themselves must make. The goal is not to automatically block every unusual message. Rather, it is about reliably detecting, stopping, and processing risky payment and data requests in a way that is auditable and traceable.

Why Business Email Compromise is so convincing

BEC is not a single piece of malware, but a family of scams. Criminals pose as a known executive, supplier, or colleague. Sometimes only a sender address is spoofed. In other cases, a real mailbox is compromised. Then the signature, earlier message threads, and internal phrasing match especially well. According to "CERT.at corresponding reports from Austrian organizations have recently become more frequent.

The attack exploits human routines more than a technical vulnerability. Typical levers are urgency, confidentiality, authority and a seemingly small deviation from normal procedure. A new IBAN, an unusual gift card order, or a request not to show the matter to anyone are meant to prompt employees to act quickly. Generative AI can create convincing variants in good German and incorporate publicly available information about roles or projects. Perfect grammar is therefore no longer a sign that there’s nothing to worry about.

What an AI audit can do – and what it can't

An internally approved AI system can structure the content of a message according to defined patterns: Is a payment being requested? Has a bank account changed? Is artificial time pressure being created? Is an established approval route being bypassed? Is there a request for passwords, personnel data, or confidentiality? Such an analysis is useful when it appears as an indication and not as a final judgment. Legitimate emails can also be unusual, and sophisticated attacks may be phrased inconspicuously.

Employees should never copy entire suspicious emails, including names, addresses, billing information, or attachments, into a publicly accessible AI tool. That can disclose trade secrets and personal data. Only a company-vetted service with clarified contracts, storage rules, and access controls is permitted. Often even a local checklist without AI is sufficient. The WKO-Guideline on legal framework conditions betont Datenschutz, Rechte Dritter und die rasche Meldung schwerer Fehler.

The secure review process in seven steps

  1. Stop action: Noch nicht antworten, keinen Link öffnen, keinen Anhang starten und keine Zahlung vorbereiten.
  2. Identify the request: Is it about money, access credentials, new master data, confidential documents, or bypassing the normal process?
  3. Verify sender technically: Carefully read the actual address, reply-to address and domain. Display name or company logo are not proof of identity.
  4. Cross-check context: Does the request match the order, contract, amount, appointment and usual role? Record any deviations in writing.
  5. Use a second channel: Contact the reportedly instructing person via a phone number already known or an internal channel – never via contact details from the suspicious message.
  6. Apply the four-eyes principle: Financial transactions and master data changes are reviewed by two separate people. One person should not both create, approve, and execute.
  7. Report incident: Forward the message to the internal IT or security team so that similar emails, rules, and affected accounts can be checked.

This procedure follows the practical recommendations of CERT.at. It prevents an AI score from being seen as a convenient substitute for identity verification. A model may, for example, indicate "high risk" and state reasons. Approval or rejection remain with the designated persons who know the business context.

A concrete example from the accounting department

An employee at a Salzburg company receives an email from a trusted supplier on Friday afternoon. The invoice looks familiar, but the bank details are said to need immediate change due to a "system migration." An internal checking tool flags three indicators: new payment details, unusual urgency, and a deviation from the usual communication channel. That is not yet proof of fraud. The employee calls the number stored in the master data system and learns that no change was requested.

The crucial protection was not the AI, but the combination of the alert, a known callback channel, and blocked changes to master data. If the model had failed or had rated the email as innocuous, the same process would need to kick in. Therefore control steps should be included in the work instructions and not only in a technical product.

Design prompts without confidential data

If a shared system is used for text analysis, the task should be narrowly defined. Instead of 'Is this email genuine?' a verifiable instruction is more useful: 'Classify the anonymized text into the categories payment request, master-data change, time pressure, confidentiality, access credentials and process deviation. Name only the passages that justify your classification. Do not make a payment decision.' Names, account numbers, phone numbers, signatures and project details are removed beforehand unless they are needed for the analysis.

The result should not be placed unreviewed into a personnel file or used in a performance evaluation. Anyone who reports suspicious messages is acting in a security-conscious manner and must not be penalized for a false alarm. Training should show that even experienced employees can be deceived. A good reporting culture rewards stopping early and asking questions.

Technical measures that support the work process

The best text analysis is of little help if accounts are reachable without multi-factor authentication or external forwards go unnoticed. CERT.at recommends modern MFA, rules against unusual access as well as SPF, DKIM and DMARC for your own domains. Automatic forwards to external addresses should be restricted and monitored. The current overview of WKO zur IT-Sicherheit bundles additional protective measures for Austrian companies.

Permissions must also match the role. Not every person who receives an invoice should be able to change supplier master data. Payment preparation and payment approval should be separated. Logs must make it possible to see who initiated, reviewed and approved a change. These organizational controls are robust against new text variants, whereas purely language-based filters can always be circumvented.

How teams train realistic scenarios

Awareness training is effective when it mirrors real roles and processes. Accounting practices a changed IBAN, the assistant an allegedly confidential instruction, HR an urgent request for employee data, and procurement a new delivery address. Each exercise ends with the same question: Which known second channel confirms the identity? The WKO guidance on phishing remind you not to disclose confidential information in response to unexpected messages and to use the usual login methods.

AI can generate variants for internal exercises, but only using fictional people, amounts, and companies. Real customer data or actual access credentials do not belong in training prompts. It must also be made clear that this is an exercise and how the results will be used. The goal is behavioral safety, not the covert assessment of individual employees.

What to do after a click or payment

No error should be covered up. After a click, a password entry, or a transfer, speed is crucial. Affected parties immediately report the incident, disconnect the device from the network if necessary, change credentials via a clean channel, and inform the bank and internal responsible parties. IT checks sessions, forwarding rules, MFA registrations, and other recipients. Evidence is secured without reopening suspicious files.

If personal data are affected, the company must assess the risk. The Austrian Data Protection Authoritydescribes when a notification is required and what information it must contain. A possible deadline does not only start once all details are known. Therefore, data protection, IT security, legal counsel, and executive management should be part of a predefined escalation path. In addition, the existing jobspot-guide on Handling AI incidents in the company.

How an AI alert remains traceable

A useful alert shows more than just a traffic-light color. It names the detected indicators, the relevant text excerpt and the scope of the assertion. “High urgency and request for changed payment details detected” is verifiable. “Fraud with 96 percent probability” sounds precise, although the number means little without appropriate test data. Teams should therefore define which specific indicators always trigger a manual review. These include new bank details, unusual payees, mismatching domain names, confidential special channels and requests to bypass controls.

Before rollout the system is tested with fabricated examples from the organisation’s own processes. The test includes real business situations without actual personal or account data: legitimate rush order, announced supplier change, forged management instruction and compromised conversation thread. Measurement is not only how many attacks are detected. It is also important how many normal operations are unnecessarily stopped and whether employees understand the explanations. Thresholds are then adjusted, documented and regularly rechecked.

Changes to the model, rules or data sources require a version identifier. If an alert is investigated later, it must be possible to see which configuration was in effect. At the same time the company should log as little email content as possible. Usually the event type, timestamp, detected risk indicators and the process decision taken are sufficient. Full messages should only be placed in a protected case archive if they are required for investigation or legal obligations.

Roles and escalation in Austrian SMEs

Even small businesses don’t need a complicated control center, but they do need clear roles. Employees report suspicious requests to a known address or phone number. A designated person checks the mail account and technical indicators. Accounting stops payments, procurement blocks master-data changes and management decides on external communication. Data-protection officers assess possible personal data links. If external IT service providers are involved, availability, responsibilities and secure handover procedures must be agreed before an incident.

A simple escalation matrix can distinguish three levels. Level one covers a suspicious request not yet carried out: report, check, confirm sender. Level two applies when credentials have been entered or an attachment executed: secure the account, isolate the device, check logs. Level three concerns payments, data exfiltration or a compromised administrative account: activate the emergency team, contact the bank and, if applicable, authorities, assess the privacy risk and continuously document decisions.

For shift and part-time teams the process must also work outside normal office hours. A management instruction in the evening must not go unchecked simply because the usual approver is not reachable. The safe rule then is: wait or escalate to a designated deputy. No legitimate business transaction should require employees to secretly bypass a central control.

Five questions before every payment or data change

A short cue card at the workstation is often more effective than a long guide. First: Does the request match the known order? Second: Have bank account details, an address, or a role been changed unexpectedly? Third: Am I being asked to act unusually quickly or confidentially? Fourth: Can I confirm the identity via an already known channel? Fifth: Has a second authorized person reviewed the process? Even a single unresolved "no" or warning signal is enough to stop.

AI can display these questions in the mail client and pre-sort available information. However, it should not invent an answer when data are missing. "Unknown" is a valuable result. The system must also not simulate a phone call or autonomously adopt new contact details from the suspicious email. Verification means using an independent source: an approved supplier directory, an existing contract, or the internal address book.

Checklist for a robust BEC process

  • There is a documented call-back procedure with pre-stored contact details for payment and master data changes.
  • The four-eyes principle and separation of duties are implemented both technically and organizationally.
  • MFA, mail-domain protection, and rules for forwarding are regularly checked.
  • AI analysis uses only approved systems and, where possible, anonymized content.
  • The model provides reasons and uncertainty, but no automatic approval.
  • Employees know a simple reporting channel and are not penalized for false alarms.
  • There are separate emergency cards for account compromise, payments, data exfiltration and data protection notifications.

Conclusion: Trust needs a second channel

AI can make suspicious patterns visible, explain formulations and structure the initial review. However, it can neither confirm the identity of an executive nor fully know the business context. Austrian companies gain security when they combine technical indicators with a callback, the four-eyes principle, clear authorizations and an open reporting culture. Those who pause under time pressure and check via a known channel are acting professionally — even if the message later turns out to be genuine.

AI note: This article was created using AI as a structuring drafting aid and was editorially reviewed. For specific security, data protection or legal questions, the relevant specialist departments should be consulted.