Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MacOS Audio #59

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from
Draft

MacOS Audio #59

wants to merge 6 commits into from

Conversation

tooxo
Copy link
Contributor

@tooxo tooxo commented Aug 12, 2024

This is a very rudimentary proof of concept, using macos audio-toolbox.
To make this remotely ready for production, I would need some help from someone who knows go (especially cgo) better than me.

@dannyherrmann
Copy link

dannyherrmann commented Aug 14, 2024

@tooxo I am in no way a go expert, and especially not a cgo expert, rather Librespot enthusiast and just trying to help/learn. I tested this branch on my M1 MacBook Air (2020) running Mac OS Sonoma 14.5. I use Zeroconf for auth and I am connecting to the go-librespot device from the Spotify client on my MacBook. Results are below it hit the not supported panic in DelayMs(). Might be way off here, but it appears we must implement this DelayMs() method using the AudioQueue API from AudioToolbox in order to calculate and return the current delay in the audio playback pipeline. I asked copilot to recommend a potential implementation of this method and it recommended calculating the delay as the difference between the current time and the time when the buffer's data is scheduled to be played. I updated this method on my local machine and was surprised to find it's now working with the updated DelayMs() function provided below. I was VERY surprised when DJ Sabrina the Teenage DJ's "Next to Me" actually starting playing on my airpods lol FYI volume control from Spotify client is not working. I hope this helps, Till.

Relevant Code below - line 257

func (out *output) DelayMs() (int64, error) {
	panic("not supported")
}

Proposed Method from copilot:

func (out *output) DelayMs() (int64, error) {
	var queueTime C.AudioTimeStamp
	err := C.AudioQueueGetCurrentTime(out.audioQueue, nil, &queueTime, nil)
	if err != 0 {
		return 0, out.alsaError("AudioQueueGetCurrentTime", err)
	}

	// Convert to milliseconds
	delay := int64((queueTime.mSampleTime / C.Float64(out.sampleRate)) * 1000)

	return delay, nil
}

Environment:

Compiler: Apple clang version 15.0.0 (clang-1500.3.9.4)
Target: arm64-apple-darwin23.5.0
Go Version: go version go1.22.6 darwin/arm64

Command Line Output:

INFO[0000] generated new device id: fff6bd8d9ceb4cc94183504a1715dfb8c533bc12 
INFO[0014] loaded track "Next To Me" (paused: true, position: 5524ms, duration: 457753ms, prefetched: false)  uri="spotify:track:16MKLKWVZPt84xOwCU56hg"
panic: not supported

goroutine 67 [running]:
github.com/devgianlu/go-librespot/output.(*output).DelayMs(...)
        /Users/dannyherrmann/go-librespot/output/driver_macos_audio_toolbox.go:257
github.com/devgianlu/go-librespot/output.(*Output).DelayMs(...)
        /Users/dannyherrmann/go-librespot/output/output.go:81
github.com/devgianlu/go-librespot/player.(*Player).manageLoop(0x1400035d9e0)
        /Users/dannyherrmann/go-librespot/player/player.go:195 +0x8b8
created by github.com/devgianlu/go-librespot/player.NewPlayer in goroutine 1
        /Users/dannyherrmann/go-librespot/player/player.go:91 +0x200
exit status 2

@tooxo
Copy link
Contributor Author

tooxo commented Aug 14, 2024

@tooxo I am in no way a go expert, and especially not a cgo expert, rather Librespot enthusiast and just trying to help/learn. I tested this branch on my M1 MacBook Air (2020) running Mac OS Sonoma 14.5. I use Zeroconf for auth and I am connecting to the go-librespot device from the Spotify client on my MacBook. Results are below it hit the not supported panic in DelayMs(). Might be way off here, but it appears we must implement this DelayMs() method using the AudioQueue API from AudioToolbox in order to calculate and return the current delay in the audio playback pipeline. I asked copilot to recommend a potential implementation of this method and it recommended calculating the delay as the difference between the current time and the time when the buffer's data is scheduled to be played. I updated this method on my local machine and was surprised to find it's now working with the updated DelayMs() function provided below. I was VERY surprised when DJ Sabrina the Teenage DJ's "Next to Me" actually starting playing on my airpods lol FYI volume control from Spotify client is not working. I hope this helps, Till.

Relevant Code below - line 257

func (out *output) DelayMs() (int64, error) {
	panic("not supported")
}

Proposed Method from copilot:

func (out *output) DelayMs() (int64, error) {
	var queueTime C.AudioTimeStamp
	err := C.AudioQueueGetCurrentTime(out.audioQueue, nil, &queueTime, nil)
	if err != 0 {
		return 0, out.alsaError("AudioQueueGetCurrentTime", err)
	}

	// Convert to milliseconds
	delay := int64((queueTime.mSampleTime / C.Float64(out.sampleRate)) * 1000)

	return delay, nil
}

Environment:

Compiler: Apple clang version 15.0.0 (clang-1500.3.9.4)
Target: arm64-apple-darwin23.5.0
Go Version: go version go1.22.6 darwin/arm64

Command Line Output:

INFO[0000] generated new device id: fff6bd8d9ceb4cc94183504a1715dfb8c533bc12 
INFO[0014] loaded track "Next To Me" (paused: true, position: 5524ms, duration: 457753ms, prefetched: false)  uri="spotify:track:16MKLKWVZPt84xOwCU56hg"
panic: not supported

goroutine 67 [running]:
github.com/devgianlu/go-librespot/output.(*output).DelayMs(...)
        /Users/dannyherrmann/go-librespot/output/driver_macos_audio_toolbox.go:257
github.com/devgianlu/go-librespot/output.(*Output).DelayMs(...)
        /Users/dannyherrmann/go-librespot/output/output.go:81
github.com/devgianlu/go-librespot/player.(*Player).manageLoop(0x1400035d9e0)
        /Users/dannyherrmann/go-librespot/player/player.go:195 +0x8b8
created by github.com/devgianlu/go-librespot/player.NewPlayer in goroutine 1
        /Users/dannyherrmann/go-librespot/player/player.go:91 +0x200
exit status 2

Thank you very much for your feedback, I implemented the delay and volume now, so i hope it at least works a little bit more flawless now.

@dannyherrmann
Copy link

@tooxo this is working perfectly! no DelayMs() panic anymore and volume control works as well. I see you used the lower level AudioObjectGetPropertyData instead of AudioQueue to query the device's inherent latency, nice. Please let me know if you need any more help testing.

@tooxo
Copy link
Contributor Author

tooxo commented Aug 15, 2024

@tooxo this is working perfectly! no DelayMs() panic anymore and volume control works as well. I see you used the lower level AudioObjectGetPropertyData instead of AudioQueue to query the device's inherent latency, nice. Please let me know if you need any more help testing.

If you want to, testing it for longer periods at a time would probably find some more smaller issues, but what I would rather need is some advice on how to test cgo code, especially searching for memory leaks and undefined behaviour.

@dannyherrmann
Copy link

@tooxo this is working perfectly! no DelayMs() panic anymore and volume control works as well. I see you used the lower level AudioObjectGetPropertyData instead of AudioQueue to query the device's inherent latency, nice. Please let me know if you need any more help testing.

If you want to, testing it for longer periods at a time would probably find some more smaller issues, but what I would rather need is some advice on how to test cgo code, especially searching for memory leaks and undefined behaviour.

I can definitely test it for longer periods of time!

re: searching for memory leaks, it looks like we might be able to use valgrind for this? Debugging Cgo memory leaks

@tooxo
Copy link
Contributor Author

tooxo commented Aug 15, 2024

@tooxo this is working perfectly! no DelayMs() panic anymore and volume control works as well. I see you used the lower level AudioObjectGetPropertyData instead of AudioQueue to query the device's inherent latency, nice. Please let me know if you need any more help testing.

If you want to, testing it for longer periods at a time would probably find some more smaller issues, but what I would rather need is some advice on how to test cgo code, especially searching for memory leaks and undefined behaviour.

I can definitely test it for longer periods of time!

re: searching for memory leaks, it looks like we might be able to use valgrind for this? Debugging Cgo memory leaks

Yes, I used valgrind before, but the problem is I only have a M1 mac, where there is no supported version unfortunately

@devgianlu devgianlu added the enhancement New feature or request label Aug 16, 2024
@devgianlu
Copy link
Owner

@tooxo What do you think is the state of this PR?

@stronk-dev
Copy link

stronk-dev commented Dec 10, 2024

Used this PR to get a dev environment setup on my Macbook, but made some changes like to volume adjustment, buffer allocation, buffering logic in general (no more ring buffer).

Will be testing: master...stronk-dev:go-librespot:feature/AudioToolbox

@devgianlu
Copy link
Owner

Used this PR to get a dev environment setup on my Macbook, but made some changes like to volume adjustment, buffer allocation, buffering logic in general (no more ring buffer).

Will be testing: master...stronk-dev:go-librespot:feature/AudioToolbox

There have been some refactors of the audio part since this PR was originally created so it's very likely that this was outdated. If you feel like you have something working feel free to open a new PR, this one may be stale.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants