linuxaudio.org

On Thu, Jun 18, 2009 at 11:16 AM, Patrick

Stinson wrote:

quoted text
> I wrote a scripting engine for a pro audio plugin by embedding a

> CPython interpreter. Since audio sequencing hosts use separate threads

> for each audio track, loading more than one instance of my plugin

> while running scripting code causes contention on Python's GIl and

> results in CPU spikes at low latencies.

>

> While CPython is more than capable of running fast enough in the audio

> thread for control-rate (MIDI) work, the GIL is killing us. Using

> process migration to move calls to CPython to daemon processes would

> take less code than forking python itself, but the scripting engine

> includes a python extension module that exposes pointers to the C++

> classes in the audio engine. This complicates things quite a bit. The

> calls look like this:

>

> size = engine.getAudioBufferSize()

> engine.loadPatchFile(SOME_PATH)

> instrument.loadAudioFile(SOME_PATH_2)

> instrument.setVolume(.5)

>

> ...where 'engine' and 'instrument' are C++ classes in the audio engine

> that I wrapped in a python extension.

>

> I think that the biggest problem for me is tackling the complexity of

> managing new real time daemon processes for each audio track, finding

> an IPC method, and also implementing a middle-ware layer that would

> allow those synchronous calls to the engine be made back to the host.

> This is the sort of overhead that one can expect when trying to use

> process migration to work around the GIL. I consider myself an

> experienced coder, but just thinking about finding a clean, rock-solid

> daemon management and IPC mechanism that works on Mac and Windows

> freaks me out.

>

> Does anyone have any comments on this topic?

>

> cheers,

> -P

> _______________________________________________

> Linux-audio-dev mailing list

> Linux-audio-dev@lists.linuxaudio.org

> http://lists.linuxaudio.org/mailman/listinfo/linux-audio-dev

>

My previous experience of getting around the GIL (not for audio stuff,

networking related) the multiprocessing module in python 2.6, or

pyprocessing before that is great. I had very good performance

improvements and the effort wasn't that extensive. The IPC /

management mechanisms  included  where enough for my needs. They are

really easy to use. For my problem performance basically scaled

linearly by number of cpus using the module. On 2-8 core machines

running many mostly io bound processes. Have you looked at stackless

as well? pyprocessing is really easy though if it works. Feel free to

give me a yell with any issues.
Loki

I am having an awesome day ;)

_______________________________________________

Linux-audio-dev mailing list

Linux-audio-dev@lists.linuxaudio.org

http://lists.linuxaudio.org/mailman/listinfo/linux-audio-dev