JACK Audio Support

Hi Glen,
Thanks a lot for your great support and this interesting idea.
That 's right that Audio is our weakness and actually, we have began to speak about this. And several great ideas have born. If Jack could be integrated in Openshot it would be enormous ( Personally, I see the potential (same if i am not an expert ) and what Jack brings us for Openshot, I have had a similar conversation with some members of the French community there is one week or two ago.

 | On the Debian side of things I could probably speak to the pkg-multimedia team about getting python-jack (pyjack) into Debian if that would help any.
It will be helpful for us and a first step for the possible integration. This feature must be easy to install and it is first by the library.

Hum. It is time for us to see more in details the Audio and we know that so it is the reason why we have began to speak about this. Perhaps it will be the main development axe for the next version.

Thanks for you and your great project. ( Are you prepare another version soon ?)
Olivier

Hi Glen! Nice to hear from you. I do not know much about Jack, which is why I have not put much time into integrating OpenShot with Jack. But I am definitely interested. Here is Jack / MLT related info from Dan Dennedy, from the MLT mailing list I found from August 2010:

------------------------------
> Is it possible to record with MLT using jack?

Jack is implemented as a MLT audio filter where the sends and receives
are optional. So, you can receive only. Also, another instance of the
filter could send only for Jack output. If you use both, then it truly
acts as a filter - e.g. sending through Jack Rack and receiving the
results.

> How does jack in MLT behave? Does it stuff like automatic
> connection/disconnection in some cases?

You can set properties on the MLT filter to tell it to setup
connections. Stereo only at the moment, I believe.

> Does it support jack transport, and to which degree?

No. MLT filters should not try to affect the position or transport.
The application which uses MLT should implement that.

> How does it behave in low latency situations?

I do not think well; no special care has been taken, e.g. avoid memory
allocations during audio filter chain processing. The Jack support was
initially added to support _prototyping_ a Jack Rack setup, which you
can save to a file. Then, for production-oriented running, you would
supply the jackrack XML file, which is processed as a stack of LADSPA
filters instead of routing through Jack. That was the workflow that
inspired the original implementation, at least. It is not necessarily
limited to that, but I think it needs testing and improvement for
general usage.

I think another issue is that MLT does not provide audio sample level
precision for audio editing operations. There is always a frame
whether there is video or not (dummy video is generated). The number
of audio samples for the duration of the frame (e.g. 40ms for 25 fps)
is stored on each frame object that propagates through the system.
With that said, if someone wants to try to improve it, I am not
opposed.
------------------------------

So, from reading this info, it would seem that MLT might not be fully ready to support Jack, without us fixing some issues. Also, I do not think that pyJack helps us any (unless I am not understanding how it works). It offers Python bindings to control Jack, but it doesn't integrate with MLT, which is needed. For example, MLT has to be able to preview video / audio using Jack, but also it has to be able to export each frame (not in real time) using the FFmpeg libavformat library. Again, I'm not that familiar with Jack, but hopefully someone with more knowledge can take this info, and start to come up with a plan. =)

Thanks Jonathan and Olivier for your prompt and informative replies :-)

I understand better now what the issues involved are and that this is largely a waste of precious development time until MLT is on board with better JACK implementation. Thanks for the clarification.

 As a plan 'B' what about LADSPA plugin integration with ALSA instead of JACK? I can't speak on Pulseaudio because AV Linux doesn't use it but most other Audio programs that don't rely on JACK use ALSA. If I'm not mistaken Cinelerra can use them. The great thing about LADSPA is it is the oldest and best known Linux plugin architecture and is present in pretty much every distrbution and many of them contain really good code and algorithms, the 'bad' thing is they rely on the host to provide the UI. Due to the higher latencies involved with ALSA vs. JACK I'm not sure how A/V syncronization would be affected obviously I'm sure that would vary with the plugin chain. Even a few select LADSPA's ie. (Compression, EQ, Reverb). would be a great start and there are several great examples of each by various LADSPA developers.

I really wish I could contribute more than dialogue on this issue, unfortunately my current Linux 'abilities' don't include python coding :-(

@Olivier - AV Linux 5.0 is in development and should be out in a couple of months, of course as always the latest incredible Openshot developments will be included when it is released.

Thanks for listening (reading) and responding, I truly hope that some avenues will open up to fortify the Audio side of things at your time and discretion. Keep up the great work guys!

@Jonathan
I suggest to invite GMaq/Glen at your future "reunion", his participation will be helpful.

@Glen
Wonderful and if you can include the same icon theme that LMDE .....................

Thanks for all.

Being the resident JACK/Audio expert (ok, expert is too strong, but I use it a lot) there are 2 different JACK issues here.

1. JACK Transport support:

For those who don't know what it is, it lets one application control the playback of another - so press play in a an audio app, and Openshot would also start playing; move the playback position in the audio app and Openshot would seek to that position.
While I haven't tried it out, I've taken a quick look at the pyJack code and I think it could work independently of MLT - pyJack would provide frame position information, this would then be used by Openshot to seek to - think of it as an invisible hand moving the playhead.

I plan on giving it a go in the next few weeks as an experiment - although I'm having plenty of fun with AVLinux 4.2 and some new synths, so I can't promise anything :-)

2. JACK Integration:

This would allow the routing of audio from Openshot to/from other JACK aware applications - this definitely would require more support in MLT, and is probably less useful in a video editor than transport support.

In addition to those, Glen also mentions LADSPA support. I've been wanting to do this for a while but haven't found the time. MLT supports this already, although in a bit of a clunky way. As well as having an xml file defining each effect, there is also a certain amount of hard-coding for each effect required, which is less than ideal. But I agree that having some of the LADSPA EQ's and Dynamics plugins available would be a great help, especially as the audio from some cameras is truly appalling.

So in short, I'll give transport and LADSPA support a go.

Getting pyJack into Debian would be handy.

Andy,

That sounds great, thanks for looking into it. I agree that JACK transport in tandem with an audio editor (ie Ardour) would open a lot of doors and allow for very streamlined workflow for things like soundtrack/scoring work and professional music video production. All work could be done with Audio and Video in their native editing environment and the final stereo mix could be imported into Openshot when the editing is finished and exported and muxed from there...correct?

I will request pyjack for pkg-multimedia, not that I have a lot of pull but they are aware of my project and have helped me in the past. If KDEnlive is looking at this issue perhaps they are also looking at getting it into Debian as well since it is their development platform.

Sounds exciting, please keep me in the loop.
Thanks! -GLEN

Hi Again,

I have put an official request report into Debian for pyjack and spoken to the maintainers, they are interested in it's potential so we will wait and see as it moves through Debian packaging channels. In the meantime having a look at the source code directly it has already been "Debianized" for Ubuntu by it's developer FalkTX who is also the maintainer of the excellent Kubuntu based 'KXStudio'. I have built some packages here which should be useable in both Debian and Ubuntu if you want to have a look Andy. Actually I built these a while ago and forgot about them when I was experimenting with Ladish...Duh!

http://www.bandshed.net/debian/python-jack_0.5.2-1~build1_i386.deb
http://www.bandshed.net/debian/python-jack-demos_0.5.2-1~build1_all.deb

Hi Glen,
Thanks a lot. Is the second packages usable on a 64 bits system ? or is it just the first ? (sorry for my stupid question ^^)

cenwen,

Yes the 'python-jack-demos' package is for all architectures and the 'python-jack' is 32bit only (I only use 32bit). Of course if you like you can simply grab the source here: http://sourceforge.net/projects/py-jack/files/py-jack/0.5.2/pyjack-0.5.2.tar.gz/download

If you want 64bit then there are Ubuntu packages here in the KXStudio PPA's I don't know what Ubuntu you are using so you will have to choose from the options in this thread: http://ubuntuforums.org/showthread.php?t=1670196

Hope this helps (sorry for my stupid answer :b)

Glen,
Thanks for your Great answer. You light my lantern. I am on Lucid for one and LMDE for the second.
I will take the KXStudio because I know the quality of is repository.

Olivier.

@ Glen - I found those packages the other day on your website, so already been looking through the examples.