Sunday, September 24, 2023

Ventura and hourly wakes

Deciphering what causes macOS Ventura to wake once every hour.

Sunday, September 24, 2023 - Sam Rowlands

Update: Sunday Sep 24 The solution for the Wake Requests, stops Wake Requests, however I have experienced Restless Mac Syndrome. So far it appears inconsistent. Also updated is more research into AOP wakes.

At the request of a customer, I started to investigate this issue. Somehow I'd completely missed it on the Ventura/Sonoma test machine, but once I had a 16" MacBook Pro M2 Pro, it was unavoidable.

wake Requests

Out of 8 wakes during the night, two show up as "rtc/Maintenance". In the Power Management messages, I can see Wake Request messages and the times match.

22:16:39 wakeRequests  Selected RTC wake request: {
	    UserVisible = 0;
	    appPID = 424;
	    eventtype = wake;
	    scheduledby = "com.apple.alarm.user-visible-com.apple.CalendarNotification.EKTravelEngine.periodicRefreshTimer";
	    time = "2023-09-19 17:08:39 +0000";
	}

On my system appPID 424, is "UserEventAgent".

The UserEventAgent utility is a daemon that loads system-provided plugins to handle high-level system events which cannot be monitored directly by launchd(8). The UserEventAgent utility should not be invoked directly.

What does the "scheduledby" message mean? My guess is that this is the event/script/plugin it runs at that time, there is a "com.apple.alarm" plugin.

A newer version of Sleep Aid will pull out these details for you.

What didn't work

Sleep Aid was unable to remove these schedules as it doesn't know the private IDs used by UserEventAgent at creation time.
Sleep Aid executed a shell script on sleep "echo "<password/>" | sudo -S pmset sched cancelall", which cleared the list, but UserEventAgent would add them back just as the Mac actually went to sleep.
Preventing the system from updating the "AutoWake.plist" did prevent new schedules being added, but caused Restless Mac Syndrome

What worked?

Executing a shell script (via Sleep Aid) to "sudo pkill -STOP UserEventAgent", then to use "pmset sched cancelall". Sleep Aid then executed a script on wake to "pkill -CONT UserEventAgent". I verified the log and there were no wakeRequest references during the sleep.

Down we go, down the rabbit hole

Lets focus on the remaining 6 or wakes that appear not be related to wakeRequests. I have data from 3 Macs, 2 on Ventura and one on Sonoma. They show different reasons but the pattern is always the same, the Mac wakes every 70 - 84 minutes or 4238 - 5073 seconds.

M2 Pro MacBook Pro shows "spu_queue_overflow_ep44", which ends up at something called "cma", related to Location Services. M1 Pro MacBook Pro shows "spu_queue_overflow_ep42", which appears to be a magnet sensor.

M1 MacBook Air shows "rtc/Maintenance", which indicates it's a scheduled wake (not event based). but... digging through the kernel logs on the M1, the kernel says it's caused by "AOP.udioPMD", by the time the wake reason is set, it shows it as "rtc/Maintenance". Always On Processor, again, my guess is this is a Power Management Unit.

Looking through the complete log, has given me clues and helped to rule out various things.

Maybe be interesting, but maybe nothing. Grabbing kIOPMSettingAutoWakeSecondsKey, kIOPMSettingAutoWakeCalendarKey, kIOPMSettingMaintenanceWakeCalendarKey & kIOPMSettingDebugWakeRelativeKey from IORootDomain on System sleep (not screen sleep), results in no values for any of these keys, in fact I now believe that these are not used at all on ARM Macs. Which means that there is another way the macOS can tell hardware when to wake.

What it is NOT

Wake for Maintenance aka PowerNap
Keep network conenctions alive aka TCPKeepAlive
Wake for network access aka WOMP / WOL
Disabling "Hey Siri" doesn't make any difference.
While spu_queue_overflow_ep44 has something to do with locationd, disabling Location Services doesn't help.
Bluetooth beaconing sets a wake time close to the actual wake time, freezing bluetoothd, causes Restless Mac Syndrome. Disabling Bluetooth (manually) doesn't do anything.
Disabling Optimized Battery Charging doesn't help. There's a "smartCharing" helper that mentions setting up wake times.
dasd was a long shot, but the M1 log showed it scheduled wakes for about the right time.
NotificationCenter says it's not scheduling wakeups.
mDNSResponder says it is not scheduling wakeups.
airportd and Wi-Fi says the wakeups are not Wi-Fi related. Manually disabling Wi-Fi doesn't change this either.
Wake Requests.
I honestly don't think it can be UserEventAgent.

Unlikely things from the log, but just in case.

"SiriNCService: (WirelessProximity) [com.apple.bluetooth:WirelessProximity] State changed to 2 from 3". I don't know what this is or what the state is. AOP is responsible for "Hey Siri", but typically identifies a Siri wake as a "Voice_trigger."
"searchpartyuseragent: (TCC) [com.apple.TCC:access] send_message_with_reply_sync(): XPC_ERROR_CONNECTION_INVALID for message 0x13285a3b0" and other errors appear in the log, just before sleep. Again, I don't know what this is or if it's capable of triggering a wake.