Substantial changes which will have a significant effect on the privacy practices and governance of most organisations will come into effect on 10 December 2026.
New rules relating to the use and transparency of disclosure of AI and automated decision-making (ADM) tools will commence on 10 December 2026. The recent hacking of Federal Government sites, including Medicare, by rogue Open AI agents, should have all businesses on high alert and mobilised to take compliance steps.
The Federal Government has become concerned that organisations are not doing enough to address this issue and has indicated its intention to exercise its enforcement powers where it considers that no action has been taken to secure the data and personal information that a business collects.
We have set out some questions which organisations should ask and steps that should be taken to ensure good governance and demonstrate compliance.
The 10 December changes
Transparency is at the heart of the reforms. Disclosure will now be required where AI or computer programs are used to make, influence, or contribute to decisions that have the capacity to affect an individual’s rights or interests.
Almost all organisations are now using different forms of AI and will need to explain in their privacy policy what kinds of (automated) decisions are being made or assisted by AI or other computer programs. Organisations should not assume that the changes won’t apply to them. As the use of AI has become so widespread, it is built into many applications that are commonly used by even the most non-technically savvy businesses.
Common examples include employees using generative AI tools to prepare/amend correspondence. A marketing team using a tool to analyse customer habits, and the adoption of online security measures, such as 2-factor authentication, facial recognition technology and CCTV. Other common uses include a recruitment platform which ranks applicants or produces rosters, or customer service systems which use AI to prioritise enquiries.
As a minimum standard, organisations must update their privacy policies to properly inform individuals where their data/personal information may be processed and affected by the use of AI. However, to do that effectively, businesses should understand (and be able to explain) what AI they are using and how that could affect the individuals who come into contact with the business.
Understanding is essential
An organisation cannot properly describe its AI and ADM practices if it does not understand them.
Management teams and Boards should/must be asking the following key AI governance questions:
- Where is AI being used across the organisation?
- What personal, confidential or commercially sensitive information is being provided to AI systems?
- Which AI capabilities are embedded in existing software?
- Are AI systems making or influencing decisions about customers or employees?
- What authority does an AI system have to access information or take action?
- What human oversight is required?
- How are AI outputs checked for accuracy and reliability?
- What happens if an AI system produces an unexpected or inappropriate result?
- Are third-party providers using business information to train or improve their systems?
- Who is responsible for approving and monitoring AI use?
These are governance questions because they concern the organisation’s risk appetite, compliance obligations and accountability. AI cannot be treated as though it is just an IT issue.
Boards and management do not need to understand the technical operation of every AI model being used. However, they do need to be clear about what systems are used and the controls and accountability measures which are in place.
AI is unpredictable
The Medicare breach highlights the challenge that AI requires proper oversight as it can produce unexpected results when given an inappropriate prompt, when it encounters information it was not expected to access, or when it is given the ability to take actions rather than simply provide information.
The Australian public was informed in August that Medicare’s Statistics Reporting Service was breached in June 2026, when an AI agent tasked with researching Australian health information gained unauthorised access to data through a Medicare portal. The Government has said that individual Medicare records were not accessed, but the incident demonstrated that an (unchecked) AI agent could move beyond the intended task and find a way around restrictions on a website.
The incident is particularly relevant because it illustrates a different kind of risk from a conventional cyberattack. It shows that an employee does not necessarily have to intend to breach a system for an AI system to produce an unexpected outcome. The combination of an AI system, an instruction or prompt, access to information and the ability to take action can create risks that were not anticipated by the person using the system.
Considered adoption of AI is critical
Organisations should identify and enforce guidelines in relation to the adoption and use of AI, to address matters such as:
- Approval – identifying which AI tools and uses are permitted.
- Data protection – establishing what information can and cannot be entered into AI systems.
- Human oversight – ensuring significant decisions are not simply left to an AI system without appropriate review.
- Security – controlling what AI tools and agents can access and what actions they can take.
- Training – ensuring employees understand both the benefits and risks of AI.
- Monitoring – regularly reviewing AI use, particularly as software providers introduce new functionality.
- Incident response – establishing what happens when an AI system produces an unexpected result, accesses information it should not have accessed or otherwise creates a privacy or security incident.
Well-designed AI and Privacy policies are an important starting point, but can’t manage the risk on their own.
What should organisations do now?
The 10 December 2026 deadline provides a practical opportunity to undertake a broader AI and privacy health check. At a minimum, organisations should:
- Review their privacy policy and ensure that it is clear and transparent about the use of AI and automated decision-making practices.
- Map current AI use – to be transparent, businesses must understand what AI is being used across the organisation, including AI functionality embedded in existing software.
- Identify AI-assisted decisions that may significantly affect customers, employees or other individuals.
- Review information flows to determine what personal and confidential information is being provided to AI systems.
- Review AI suppliers and contracts, including data retention, security, training and access arrangements.
- Establish clear AI governance, including approval processes and due diligence, responsibility, human oversight and monitoring.
- Train employees on appropriate AI use, including what information can, or must not, be entered into AI tools.
- Regularly review the framework, because AI capabilities and the risks they present are changing rapidly. Each new AI tool should be assessed, so this can’t be set and forget.
Children's Online Privacy Code
The second limb to the reform is the commencement of the Children’s Online Privacy Code, also from 10 December 2026.
We have covered those changes in separate articles, which can be found here.
What to do now
The December reforms should not be viewed as simply a privacy policy update.
They should be the catalyst for organisations to map their AI use and establish a governance framework that is capable of keeping pace with technology, which is then reflected in their privacy policy.
Organisations should undertake an AI health-check to identify the AI tools being used, the purpose for which they are being used, the information provided to them, whether third-party providers are involved and whether AI is making or influencing decisions about individuals.
As a minimum standard, organisations must update their privacy policies to properly inform individuals where their data/personal information may be processed and affected by the use of AI.
If you would like assistance to strengthen your governance framework, understand and map your use of AI and review your AI policy or address your privacy policy, supplier arrangements or broader compliance obligations, please contact Andrew Nicholson at anicholson@mullinslawyers.com.au.
