Private Cloud Compute: Can AI in the Cloud Keep a Secret?

You have a letter you need translated. It is from your doctor, it concerns the thing on your shoulder that you have been ignoring since Easter, and it is written in a language you cannot read. There is a translator in town. The translator, naturally, will need to read the letter. That is rather the point of a translator.

So you get careful. You ask for a translator who works in a room with no windows, no filing cabinet, no notepad and no second door. You ask that a locksmith change the lock every night and throw the old key into a fire. You ask that the room was photographed brick by brick when it was built, with an independent witness in the corner holding a clipboard. You ask that its floor plan be pinned to the town noticeboard, in ink, on a numbered page. You send the letter through a courier who knows your face but never sees the letter, and pay with a stamp that proves somebody paid without saying who. Finally, you check the translator's badge against the noticeboard. If it does not match, the letter goes home with you.

Your friends think you have lost it. Your aunt leaves pamphlets about "letting go" on your kitchen bench.

And yet on Monday (Tuesday morning, from where I sit in Tasmania, which is when all the interesting American things happen), Apple announced that it had built more or less that room. Every one of those demands maps to a real part of a system called Private Cloud Compute, and Apple's security engineers spent several thousand words explaining how it works.

So put the pamphlets down. Wanting all of that is not paranoia. It is a very good list of requirements.

At its developer conference on 10 June, Apple introduced Apple Intelligence: writing tools, summaries, a smarter Siri that knows about your messages and your mum's flight. Many of its models "run entirely on device", but "more complex requests" will go to "larger, server-based models" in Apple's data centres. Apple also says that "Independent experts can inspect the code that runs on Apple silicon servers to verify privacy". And, as a separate door, Siri will be able to hand your question to ChatGPT.

Elon Musk threatened to ban Apple devices at his companies. The cryptographer Matthew Green, who has spent a career poking holes in this sort of thing, started his thread with the most honest sentence anybody wrote that day: "I'm still trying to work out what I think about this."

Me too, so I read the lot. We will start with why a cloud AI has to read your letter at all, then follow Apple's design down to the badge check and the double envelope. Then the part the keynote skipped: none of it is running yet, so I will sort what is enforced from what is promised, before we reach the door marked ChatGPT and a list of questions to send any AI vendor before your next cup of tea.

Let's get into it.

Part 1: Why the Cloud Has to Read Your Letter

You already know three states a secret can be in. A diary in a locked drawer. A letter in a sealed envelope. And a letter in a translator's hands, which is not sealed at all, by design.

Security people call these data at rest (the drawer), data in transit (the envelope) and data in use (the translator). The first two are broadly solved: your phone encrypts its storage, and its connections are wrapped in TLS, the padlock in your browser bar. The third is the hard one, and AI depends on it.

The postman and the translator

The gold standard for privacy in a messaging app is end-to-end encryption: your phone scrambles the message with a key that only the recipient's phone holds, so the company in the middle carries a sealed box it cannot open. Apple's PCC post explains why it works: "With services that are end-to-end encrypted, such as iMessage, the service operator cannot access the data that transits through the system. One of the key reasons such designs can assure privacy is specifically because they prevent the service from performing computations on user data."

End-to-end encryption is private because the server does nothing with your data. A postman does not need to read the letter, so you can seal it.

A language model is a translator. To summarise your email, it has to read your email. Apple says so plainly: the data centre hardware "requires unencrypted access to the user's request and accompanying personal data. That precludes the use of end-to-end encryption". The one exception is homomorphic encryption, which lets a server compute on data while it stays encrypted, like a translator working through a letter inside a sealed glovebox. Adelin Travers of the security firm Trail of Bits, who wrote up PCC from a machine-learning angle today, admits that on a first read he took Apple's introduction to mean it had achieved end-to-end encryption for machine learning, which would need exactly this. It had not. Homomorphic encryption, he writes, "is still considered too slow to be deployed at the cloud scale and in complex functions like ML". So Apple's servers decrypt your request and read it. Everything that follows is about what happens around that moment.

Why not keep it all on the phone?

Data that never leaves your phone cannot leak from a data centre, which is why Apple has long preferred the device. The trouble is that the phone is small. Apple's machine learning team published a technical overview of its models the same day. The on-device model has about 3 billion parameters, the numbers a model learns in training. Apple squeezes each one down to an average of 3.5 bits, using a mix of 2-bit and 4-bit values,¹ instead of the usual 16. The result produces text on an iPhone 15 Pro at about 30 tokens (pieces of words) a second.

That is a ceiling. A model that must fit beside your photos and run on a battery can only be so big. The server model is described only as "larger", which is Apple for "not telling you". Matthew Green put the gap simply: phone neural hardware is improving, but "it's not improving fast enough to take advantage of all the crazy features Silicon Valley wants from modern AI". For the hard requests, the letter has to leave the house.

The trouble with "trust us"

Every cloud AI provider says something reassuring about your data. Apple's post explains, better than most privacy advocates do, why those reassurances are usually hollow. It names three problems.

Promises you cannot check. "If a cloud AI service states that it does not log certain user data, there is generally no way for security researchers to verify this promise". Apple's example is a TLS-terminating load balancer, the receptionist at the front of a data centre who opens the encrypted envelope to decide which desk it goes to. During a troubleshooting session, Apple writes, that receptionist "may end up logging thousands of user requests wholesale".

Software you cannot see. Even with published code, there is "no widely deployed way" for your device to confirm the service runs "an unmodified version of the software that it purports to run". You can read the menu, not see into the kitchen.

Staff with master keys. Site reliability engineers (SREs) keep the servers alive at 3 am, and when something breaks, they use "highly privileged access to the service, such as via SSH", a remote command line with the keys to everything. Apple notes that "criminals such as ransomware operators routinely strive to compromise service administrator credentials precisely to take advantage of privileged access interfaces".

Nobody means harm. Somebody copies the database to a laptop "just for now", and "just for now" lasts until the laptop is stolen from a car in Sandy Bay.²

Apple contrasts this with your phone, where "Apple retains no privileged access": its file encryption "cryptographically prevents Apple from disabling or guessing the passcode". PCC is an attempt to make we could not get in even if we wanted to true of a server.

Part 2: Five Promises, in Apple's Words

max_tokensPOST./v1/messages

Apple sets out five requirements for PCC. Each is a precise claim, and precise claims can be checked. Here they are, with my translations.

1. Stateless computation on personal user data. Your data must "never be available to anyone other than the user, not even to Apple staff", and "must not be retained, including via logging or for debugging". Translation: the translator forgets.
2. Enforceable guarantees. The guarantees must be "entirely technically enforceable", not dependent on outside parts such as that receptionist. Translation: the room's walls do the work, not the translator's good character.
3. No privileged runtime access. No interface that lets Apple's own reliability staff "bypass PCC privacy guarantees, even when working to resolve an outage". Translation: there is no handle on the outside of the door, even for the night watchman.
4. Non-targetability. An attacker should not be able to go after one particular person's data "without attempting a broad compromise of the entire PCC system". Translation: nobody can find out which room your letter went to.
5. Verifiable transparency. Researchers must be able to check that the software in production "is the same as the software they inspected". Translation: the floor plan is on the noticeboard, and your phone checks it.

Apple calls this "an extraordinary set of requirements", and for once the marketing word is fair.

The hardware: a phone that thinks it is a server

The building block is a PCC node, an Apple silicon server with the same security hardware as an iPhone: the Secure Enclave (a small, separate processor that holds keys and never lets them out) and Secure Boot (a chain of checks that refuses to start any software Apple has not signed). On top runs a new operating system, "a hardened subset of the foundations of iOS and macOS" cut down to the job of running a language model.

The interesting part is what Apple took out: "remote shells and system introspection and observability tools", the things every data centre admin relies on. The software that runs the model is written in Swift, Apple's own programming language.

Stateless: forgetting on purpose

You already know two ways to destroy a document. You can shred every copy, including the one somebody left on the printer. Or you can write it in a cipher and burn the only key, and let the pages sit there as noise.

The second method is cryptographic erasure, and it is how PCC forgets. In Apple's words, "the Secure Enclave randomizes the data volume's encryption keys on every reboot and does not persist these random keys". Reboot the node and everything it wrote to disk becomes unreadable, by anyone, including Apple. That is the locksmith from the introduction, throwing the old key into the fire every night.

Between reboots, Apple says the software deletes each request's data when the reply goes back, and the memory used to handle user data is "periodically recycled to limit the impact of any data that may have been unexpectedly retained". Note the word unexpectedly. The design assumes bugs will happen and limits how long a bug can keep anything, which I find more reassuring than a promise of no bugs.

Then there are logs. A normal server keeps a running diary, and diaries are where secrets go to be found later. PCC, says Apple, "doesn't even include a general-purpose logging mechanism". Only "pre-specified, structured, and audited logs and metrics can leave the node". In plain terms: instead of a diary, the translator fills in a form with fixed boxes (say, requests handled and errors), and there is no box for "what the letter said".

No back door for the night shift

The fix for the master-key problem is blunt: there is no master key. There is no remote shell or interactive debugger, and the nodes "cannot enable Developer Mode". You cannot sneak a tool in later, either, because every piece of code the node runs must be on a list (a trust cache) signed by Apple and loaded by the Secure Enclave "such that it cannot be changed or amended at runtime". The node even refuses JIT (just-in-time compilation, where a program writes new machine code while it runs), because new code at runtime is exactly what an attacker wants.

This is a real cost. When a PCC node misbehaves at 3 am, the on-call engineer cannot log in and look around; they get the form with the fixed boxes. Trail of Bits notes the downside: limiting logs this way "could increase ML model risks", because the team may not see enough to notice when the model starts doing worse on new kinds of input. Apple chose privacy over operability, knowingly.

Part 3: Checking the Translator's Badge

So far, this is how Apple says it built the room. How would you know? This is the new part, and the one I would most like every cloud provider to copy.

A fingerprint for software

You download an installer, and the website lists a "checksum" beside it, a long string you can compare with your copy if you are the conscientious sort. That string is a hash: a short fingerprint of a file. Change one byte of the file and the fingerprint changes completely. When a computer records the hash of each piece of software as it starts, that record is called a measurement, and a computer that starts only measured, approved software has done a measured boot.

Now add a witness. Suppose the machine's own security chip signs a statement, "I booted the software with this measurement", with a private key that is sealed inside the chip and cannot be copied. That signed statement is an attestation. It lets a remote machine prove what software it is running, rather than merely claim it.

PCC nodes do this, with Secure Enclave keys that Apple says "cannot be duplicated or extracted". Attestation is not new, and Apple says so: "some specialized mechanisms exist, such as Intel SGX and AWS Nitro attestation".³ What is new is what Apple does with it.

Born under witness

An attestation is only as good as the key that signs it; a fake node with a fake key could attest to anything. So Apple's process starts at the factory. Before each server is sealed, Apple says, it will "inventory and perform high-resolution imaging of the components of the PCC node", then activate a tamper switch. In the data centre, several Apple teams revalidate them, "monitored by a third-party observer not affiliated with Apple". Only then is a certificate issued for that node's keys, and "the user's device will not send data to any PCC nodes if it cannot validate their certificates".

That is the room photographed brick by brick, with a witness in the corner. I made up the clipboard.

The noticeboard

Attestation tells your phone what software a node runs, not whether that software is any good. For that, somebody has to look, and everybody has to be looking at the same thing.

Picture a ledger in a town hall, in ink, with numbered pages. Anyone may read it, new entries go at the end, and nobody can tear out a page without the gap showing. Because everyone reads the same ledger, the clerk cannot show you one version and your neighbour another.

That ledger is a transparency log, and the web already relies on them. Certificate Transparency, described in RFC 6962 in 2013, keeps "publicly auditable, append-only, untrusted logs" of the certificates that secure websites. Untrusted is the clever word: nobody needs to trust the log's operator, because the log is built from a Merkle tree, a structure of hashes of hashes where one short value at the top fingerprints every entry below it. If a log "attempts to show different things to different people", the RFC says, this "can be efficiently detected by comparing tree roots and consistency proofs".

Apple already runs one at enormous scale. In October 2023 it described iMessage Contact Key Verification, which lets your phone check that it is using your friend's real keys. Apple says its log "receives more than two billion entries per week". The PCC post points to the same design: once a software release is signed into the log, "it cannot be removed without detection".

Apple promises to publish the measurement of every production software build in the log, and to make the matching images available "for independent binary inspection — including the OS, applications, and all relevant executables". Software will be published "within 90 days of inclusion in the log, or after relevant software updates are available, whichever is sooner". It will also publish "an official set of tools for researchers", "periodically" release "a subset of the security-critical PCC source code", and pay bounties for problems.

The handshake

Now put the pieces together, because this turns a promise into a check.

When Apple Intelligence decides a request needs the cloud, your device builds the request: "the prompt, plus the desired model and inferencing parameters". It gets a list of nodes with their attestations. Then, Apple says, "the user's device will wrap its request payload key only to the public keys of those PCC nodes whose attested measurements match a software release in the public transparency log". A node running anything not in the log never gets the key. The load balancers that pass the request along "do not have the keys required to decrypt the user's request" at all.

stop_reasoneval:.pass

That is the badge check from the introduction, and what matters is who enforces it. In a normal cloud service, the provider promises not to run secret software, and you hope. Here, your own device refuses to talk to it. To slip in a version that copies requests, Apple would have to publish it in the log, where anyone could find it. That is why Apple's newsroom can say devices "do not talk to a server unless its software has been publicly logged for inspection".

page_viewtool_use

What the noticeboard can't tell you

In 1983, Ken Thompson, co-creator of Unix, accepted computing's biggest prize, the Turing Award, with a lecture called Reflections on Trusting Trust, printed the following year. He showed how a compiler could be rigged to plant a back door in the programs it built, and hide the rigging from anyone reading its source. His conclusion: "You can't trust code that you did not totally create yourself." He added that "as the level of program gets lower, these bugs will be harder and harder to detect", right down to "hardware microcode".

PCC does not escape this, and Apple does not claim it does. The log proves that the software running is the software published. It does not prove that the published software is innocent. Somebody has to look, and what Apple promises is mostly binaries, the compiled machine code, not the human-readable source. Green spotted this straight away:

Of course, knowing that the phone is running a specific piece of software doesn’t help you if you don’t trust the software. So Apple plans to put each binary image into a “transparency log” and publish the software.

But here’s a sticky point: not with the full source code.

— Matthew Green (@[email protected]), 11 June 2024 on Mastodon

(He means the server, not the phone.) In his next post he notes that researchers will "have to reverse-engineer the binaries to see if they're doing unexpected things", and calls that "a little suboptimal".

Apple does sweeten it: the images will include the Secure Enclave's firmware (sepOS) and the iBoot bootloader "in plaintext", "a first for any Apple platform". Still, reading binaries is slow, skilled work, and a noticeboard only helps if people with clipboards actually read it.

Part 4: Hiding Who Asked

utm_source=SELECT

Meet Margit, a (made-up) journalist. A well-funded attacker wants to read what she asks her phone: just hers. In a typical cloud service, Apple notes, every application server holds credentials for the whole database, "so a compromise of a single application server is sufficient to access any user's data".

PCC's answer is non-targetability: even an attacker who breaks into a node cannot make Margit's requests go to it. Apple calls the method target diffusion, and it has three tricks.

Trick one: two envelopes

To send a letter that cannot be traced to you, put it inside a second envelope, addressed to a friend, and ask the friend to post the inner letter on. The friend knows it came from you but cannot open it. The recipient can read it but sees only your friend's postmark.

On the internet, that is Oblivious HTTP (OHTTP), standardised this January as RFC 9458. Your device encrypts the request for the destination and sends it through a relay. The standard says the arrangement "ensures that Oblivious Gateway Resource never sees the Client's IP address and that the Oblivious Relay Resource never sees plaintext HTTP message content". The relay knows who you are, the destination knows what you said, and neither knows both.

If you read my piece on DNS over TLS, you will recognise the problem: encryption hides what you say from people along the way, but whoever answers still sees who is asking. A VPN moves that knowledge rather than removing it: the VPN provider now sees everything. OHTTP splits it between two parties who are not meant to share notes.

For PCC, Apple says requests go through "an OHTTP relay — operated by a third party — which hides the device's source IP address before the request ever reaches the PCC infrastructure". So "an attacker would have to compromise both the third-party relay and our load balancer" to steer traffic by IP address. Apple has not said who runs the relay.

Trick two: a ticket without a name

Hiding your IP address creates a new problem. If Apple cannot see who is asking, how does it stop the whole internet using its expensive servers for free? It needs proof that the request comes from a real Apple device, without learning which device.

This is the job of a blind signature. Picture a document in an envelope lined with carbon paper. A notary signs the outside without opening it, and the carbon copies the signature onto the document inside. Later, the notary can confirm the signature is genuine, but cannot tell when they signed it or for whom.

The modern version, RSA blind signatures, is written up in RFC 9474 from October 2023. One of its authors works at Apple, and the RFC lists Apple's iCloud Private Relay among its uses.⁴ The client blinds a message, the server signs it, the client unblinds it, and in the RFC's words, "the server learns nothing, whereas the client learns sig". Apple's PCC requests carry "a single-use credential, based on RSA Blind Signatures, to authorize valid requests without tying them to a specific user".

The rest of the outer label, the metadata, "leaves out any personally identifiable information about the source device or user". That is the stamp in the introduction: proof that somebody paid, without saying who.

Trick three: write to a few rooms, not all of them

Last, your device encrypts its request only to a small set of ready nodes that the load balancer suggests, not to every node. Because the load balancer has "no identifying information about the user or device for which it's choosing nodes, it cannot bias the set for targeted users". If one node is compromised, Apple says, "it would not be able to decrypt more than a small portion of incoming requests". The load balancer's choices are also "statistically auditable".

Put the three tricks together and follow Margit's request. The relay sees her IP address and a sealed blob. The load balancer sees a one-time token and routing details. The node reads her words, has no idea they are hers, and forgets them.

aria-labelgtm.js

Where the split can leak

One gap no envelope can close, and Trail of Bits names it: all of this hides who you are in the metadata, not what you put in the letter. If Margit asks the model to "summarise this email from my editor about my source in the ministry", the node reads that sentence. In Travers' words, "there are virtually no ways of filtering out all PII in the body content sent to the LLM". The design's answer is that the node forgets and cannot be targeted. That is strong, but it is not the same as never knowing.

Part 5: Guarantee, Promise or Hope?

As of today, 15 June 2024, Private Cloud Compute is not running. Apple Intelligence arrives "in beta" this (northern) autumn,⁵ "in U.S. English", on the iPhone 15 Pro and Pro Max and on iPads and Macs with M1 or later. The images, the log and the tools are promised "When we launch Private Cloud Compute". Apple calls its post "an initial technical overview", with a deeper one to come "after PCC becomes available in beta".

So notice the tenses. The newsroom says independent experts "can inspect the code". The security blog says they will be able to. Only one of those describes this week. Here is how I sort the claims.

Enforced by the design, once it ships

Here your device, or the hardware, does the enforcing:

• Your device sends a request only to nodes whose measured software is in the public log.
• The node's storage keys are random on each boot and never saved, so a reboot erases everything.
• There is no shell, no debugging mode and no way to load new code.
• The relay and the load balancer cannot read your request.

If it works as described, none of these depend on anyone's good behaviour on the day.

Promised, and checkable later

These depend on Apple doing what it said:

• Publishing every production image within 90 days, with the tools to inspect them.
• Publishing a subset of the source code "periodically".
• Paying bounties for privacy problems.
• Running a manufacturing and validation process with a third-party observer.

Each will be visible when it happens, or visibly missing if it does not, which already beats an ordinary privacy policy.

Not addressed, or not yet

Who decides when your data leaves the phone? Apple Intelligence does. Green could find no notice or switch:

As best I can tell, Apple does not have explicit plans to announce when your data is going off-device for to Private Compute. You won't opt into this, you won't necessarily even be told it's happening. It will just happen. Magically.

I don't love that part.

— Matthew Green (@[email protected]), 11 June 2024 on Mastodon

Inference or training? Apple's post is all about inference, a model answering a request. Trail of Bits points out that "stateless" only holds for inference: if PCC were used to train models, data would be kept, because models "retain data encoded in their weights". Apple's model team says it does "not use our users' private personal data or user interactions when training". That is a policy, not something the log can show.

Who holds the root keys? The chain of trust ends at Apple. As Trail of Bits puts it, "Apple effectively owns the attestation keys". The observer and the log make cheating harder to hide, but the certificates are Apple's.

Is the hardware sound? Secure enclaves have a mixed history. In 2018, an attack called Foreshadow let an attacker "read the entire SGX enclave's memory contents" on Intel's version.⁶ Closer to home, Trail of Bits says LeftoverLocals, a GPU flaw its team disclosed this year that can let a program listen in on another's language-model output, still affects Apple's M2 chips "at the time of writing". Non-targetability may limit the damage, but this hardware is new to the job, and new hardware has new bugs.

Green's thread lists the same worry: "there are so many invisible sharp edges that could exist in a system like this. Hardware flaws. Issues with the cryptographic attestation framework. Clever software exploits."

On the device is not the same as private

One more contrast from this week. Microsoft's Recall, for its new Copilot+ PCs, periodically takes a snapshot of your screen and analyses it on the device. No cloud at all. On 7 June Microsoft made it opt-in, "off by default", with snapshots decrypted "just in time" after a Windows Hello sign-in. Yesterday it pulled Recall from the 18 June launch and moved it to the Windows Insider Program first.

Recall was entirely local and still needed an opt-in and extra encryption, because it kept everything. PCC is in the cloud and is designed to keep nothing. Where your data is processed is one question. What is kept, and who can reach it, decides whether it is private, and PCC answers that with engineering rather than a policy page.

Part 6: The Door Marked ChatGPT

Separately from PCC, Apple is adding ChatGPT to Siri and its writing tools, "powered by GPT-4o", "later this year". When Siri thinks ChatGPT could help, Apple says, "Users are asked before any questions are sent to ChatGPT, along with any documents or photos". OpenAI's announcement describes the same protections: "requests are not stored by OpenAI, and users' IP addresses are obscured". You can use it without a ChatGPT account. If you connect your account, "their data preferences will apply under ChatGPT's policies".

Within hours, this happened:

If Apple integrates OpenAI at the OS level, then Apple devices will be banned at my companies. That is an unacceptable security violation.

— Elon Musk (@elonmusk), 10 June 2024 on X

Set aside who posted it. The post blends two things this article has worked hard to separate.

PCC is Apple's own servers, with attestation and the public log. The ChatGPT door is a hand-off to another company's servers, with none of that. "Requests are not stored" is a contractual promise between two companies, not a property your device enforces. Your IP address is "obscured", but your question, and any photo or document you agreed to send, arrives in a form OpenAI's models can read.

What makes it tolerable is consent. As described, ChatGPT is not wired in silently; Siri asks each time. That is the mirror image of Green's complaint about PCC: the less private path is the one you are asked about, and the more private path just happens. So the prompt matters. "Send this to ChatGPT?" is a privacy decision, and it is worth teaching people to read it as one.

Green made the comparison that I keep coming back to:

Wrapping up on a more positive note: it's worth keeping in mind that sometimes the perfect is the enemy of the really good.

In practice the alternative to on-device is: ship private data to OpenAI or someplace sketchier, where who knows what might happen to it.

— Matthew Green (@[email protected]), 11 June 2024 on Mastodon

In his next post, he called PCC "a real commitment by Apple not to 'peek' at your data", and "a big deal". I agree. I also notice that Apple shipped both doors in one keynote: the most carefully argued cloud privacy design I have read, and a hand-off that runs on the ordinary kind of trust. Apple has made privacy a selling point for years; this week it backed that with engineering for one door, and a consent prompt for the other.

What to Do This Week

GPC:.1await

Nothing here ships until later this year, and only in US English to start. That gives you time.

If you buy AI services: send your vendor five questions

Apple's five requirements make a good checklist. Turn them into an email to any AI vendor that will see personal data, from a website chatbot to a "smart" feature in your CRM:

1. Retention. What happens to our data after you return the response? Is it logged, and where? Is any of it used for training, and can we turn that off in the contract?
2. Privileged access. Can your staff read our requests during an outage? Through which tools, and who approves it?
3. Verification. Can we, or an independent auditor, check that the software you run is the software you describe? Do you offer any form of attestation, such as the AWS Nitro or Intel SGX mechanisms Apple mentions?
4. Targeting. If one of your servers is compromised, how much of our data is exposed: one request, one customer, or everyone's?
5. Third parties. Which other companies see our data on the way (relays, model providers, logging services), and what does each one see?

Expect most answers to fall short of PCC. The point is to learn which promises are enforced by design and which are just written down.

If you manage Apple devices for an organisation

Apple's announcement does not describe any admin controls for Apple Intelligence, PCC or the ChatGPT hand-off, so I will not guess. Ask your Apple representative what controls will exist, and specifically whether you can turn off the ChatGPT hand-off on managed devices, because that is where data leaves Apple's design.

If you are responsible for privacy

Update your data map now. A request to summarise an email may soon go from staff devices to Apple's servers, and a ChatGPT hand-off sends it to OpenAI as well: two recipients, two kinds of protection. If you work under the Australian Privacy Act, the changes being proposed for it make that kind of data mapping more important, not less.

If you build systems

You do not need Apple's budget to borrow its best ideas:

• Delete the diary. Replace free-text logging of user content with structured logs that have fixed fields and no room for the content itself.
• Remove the master key. Ask who can open a shell on the machines that handle personal data, and whether they need to.
• Split the knowledge. OHTTP (RFC 9458) is a published standard. If you collect analytics or telemetry, a relay can separate who sent it from what it says.
• Separate proof of payment from identity. Blind-signature tokens (RFC 9474) can rate-limit or authorise requests without tying them to a user.

If you just own an iPhone

When Siri asks whether it can send something to ChatGPT, read the question. It is the one moment in this system where you are asked.

Final Thoughts

Back to the translator in the room with no windows. On Monday, Apple described that room in more detail than anybody has described a cloud AI service: the key in the fire, the witness, the two envelopes, the stamp without a name, and the badge your phone checks against the noticeboard. Matthew Green summed it up: "if you gave an excellent team a huge pile of money and told them to build the best 'private' cloud in the world, it would probably look like this."

The one thing to remember is the difference between a check and a promise. PCC's best idea is that your device enforces the important parts: it will not talk to software that is not on the public noticeboard. As of this week, though, the noticeboard is empty, the room has not opened, and the researchers who will read the floor plans have not seen them yet. Until they have, it is the most carefully argued promise in cloud computing, and still a promise.

Your next step is the five questions. Send them to one vendor this week, and see how many answers describe something enforced rather than something promised.

As for me, my own translator is a teapot, which keeps nothing between pots. Mostly. Now, if you'll excuse me, I think my tea is ready.

Notes

1. A 2-bit number can hold only four distinct values, and a 4-bit number sixteen. Apple calls its method "low-bit palletization", and says that with the mix, plus small add-on modules called LoRA adapters, the model reaches "the same accuracy as the uncompressed models". ↩

2. Sandy Bay is a leafy suburb of Hobart, and I have no evidence that any laptop has ever been stolen there. I needed a car, and it has nice ones. ↩

3. Intel SGX (Software Guard Extensions) puts code and data in protected regions of memory, called enclaves, on Intel processors. Apple names SGX and AWS Nitro attestation as "specialized mechanisms" that already exist, as an aside to its main point: there is "no general mechanism" for researchers to check what a cloud service is really running. ↩

4. RFC 9474 credits the idea to the cryptographer David Chaum, who introduced blind signatures in 1983 "for untraceable payments", which is to say digital cash that the bank could vouch for without knowing who spent it. ↩

5. Apple says "this fall". In Hobart that is spring, which is when the apple blossom comes out, so the timing has a certain neatness, if not a certain date. ↩

6. Foreshadow was disclosed in August 2018 by two independent research teams, and presented at the USENIX Security Symposium that year. The lesson for PCC is general: a secure enclave is still a piece of hardware, and hardware has bugs. ↩