Re: [LAD] A question about audio file interfaces

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: Fons Adriaensen <fons@...>
Cc: Linux Audio Developers <linux-audio-dev@...>
Date: Sunday, December 1, 2013 - 2:08 pm

--089e012282c884c44b04ec79998c
Content-Type: text/plain; charset=ISO-8859-1

On Sun, Dec 1, 2013 at 6:48 AM, Fons Adriaensen wrote:

>

linus has always insisted (rightly or wrongly) that user-space code should
assume omnipotence and beneficence on the part of the kernel with respect
to I/O. there are plenty of people who don't agree with him, hence things
like the madvise() call. but i think that it is a pretty good bet for most
cases. in ardour, the purpose of the disk i/o thread (the "butler") is
primarily to organize buffering within user space based on observed (but
very occasional) delays in I/O. it is not attempting to do better than the
kernel in handling read-ahead and in fact for quite a lot of situations,
ardour really does rely on the kernel's own buffering to do a good job. we
only buffer "more" because of the delays that have been seen over the
years, which suggest that is expedient to not rely on data being available
from read() within any reliable time frame less than a few seconds.

>

several years ago, linus told me in no uncertain terms not to use mmap for
disk i/o. his perspective is that even though it works, the kernel is not
developed or optimized with this in mind, and that normal streaming i/o
interfaces like read/write will always be the right path.

if you use stdio, you add an additional, hidden layer of buffering that
really provides very few benefits given that you will likely be
reading/writing fairly large chunks already.

read and write are certainly my preference, although to handle the
possibility of multithreaded access to the same file descriptor, pread and
pwrite are attractive.

> * any point in implementing things like gather/scatter, or

the kernel provides internal implementations of this, as does the firmware
on almost any storage device. probably not worth reimplementing (especially
given that you don't have any special knowledge about the device)

> * for multichannel: offer de-interleaved access, sparse

seems application-centric. some cases would need this some would not.

i would still try to add support for this format to libsndfile, so as to
gain all the benefits it offers from an application perspective.

>

--089e012282c884c44b04ec79998c
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Sun, Dec 1, 2013 at 6:48 AM, Fons Adriaensen &=
lt;fons@linuxaudio=
.org
> wrote:

No, it's just about the code to read/write the files - what
it should do or not do in order to make it perform well with
'smart' disk access threads which will have there own ways to
organise buffering. If two (or more) such schemes are used on
top of each other the result could be worse than with just
one of them.linus has always insisted =
(rightly or wrongly) that user-space code should assume omnipotence and ben=
eficence on the part of the kernel with respect to I/O. there are plenty of=
people who don't agree with him, hence things like the madvise() call.=
but i think that it is a pretty good bet for most cases. in ardour, the pu=
rpose of the disk i/o thread (the "butler") is primarily to organ=
ize buffering within user space based on observed (but very occasional) del=
ays in I/O. it is not attempting to do better than the kernel in handling r=
ead-ahead and in fact for quite a lot of situations, ardour really does rel=
y on the kernel's own buffering to do a good job. we only buffer "=
more" because of the delays that have been seen over the years, which =
suggest that is expedient to not rely on data being available from read() w=
ithin any reliable time frame less than a few seconds.
=A0

* use mmap, read/write or stdio ?sever=
al years ago, linus told me in no uncertain terms not to use mmap for disk =
i/o. his perspective is that even though it works, the kernel is not develo=
ped or optimized with this in mind, and that normal streaming i/o interface=
s like read/write will always be the right path.
if you use stdio, you add an additional, hidden layer of buf=
fering that really provides very few benefits given that you will likely be=
reading/writing fairly large chunks already.read and wr=
ite are certainly my preference, although to handle the possibility of mult=
ithreaded access to the same file descriptor, pread and pwrite are attracti=
ve.
=A0
* any point in implementing things like gather/scatter, or
=A0 async access ?the kernel provides =
internal implementations of this, as does the firmware on almost any storag=
e device. probably not worth reimplementing (especially given that you don&=
#39;t have any special knowledge about the device)
=A0
* for multichannel: offer de-interleaved access, sparse
=A0 channel sets, etc. ?seems applicat=
ion-centric. some cases would need this some would not.i=
would still try to add support for this format to libsndfile, so as to gai=
n all the benefits it offers from an application perspective.
=A0

I have various use cases for such a library, one of them being
to replace the private .ald format used by Aliki and some other
apps. But the code should also work efficiently with e.g. a DAW
having maybe a hundred files open at any time, and multiple
streams from the same file.

Ciao,

--
FA

A world of exhaustive, reliable metadata would be an utopia.
It's also a pipe-dream, founded on self-delusion, nerd hubris
and hysterically inflated market opportunities. (Cory Doctorow)

_____________________________=
__________________
Linux-audio-dev mailing list
Linux-audio-dev@lis=
ts.linuxaudio.org

http://lists.linuxaudio.org/listinfo/linux-audio-dev

--089e012282c884c44b04ec79998c--

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
[LAD] A question about audio file interfaces, Fons Adriaensen, (Sat Nov 30, 10:57 pm)
Re: [LAD] A question about audio file interfaces, Fred Gleason, (Sun Dec 1, 3:42 am)
Re: [LAD] A question about audio file interfaces, Erik de Castro Lopo, (Sun Dec 1, 1:43 am)
Re: [LAD] A question about audio file interfaces, Devin Anderson, (Sun Dec 1, 12:32 am)
Re: [LAD] A question about audio file interfaces, Fons Adriaensen, (Sun Dec 1, 11:48 am)
Re: [LAD] A question about audio file interfaces, Paul Davis, (Sun Dec 1, 2:08 pm)
Re: [LAD] A question about audio file interfaces, Fons Adriaensen, (Sun Dec 1, 4:49 pm)
Re: [LAD] A question about audio file interfaces, Paul Davis, (Sun Dec 1, 4:58 pm)
Re: [LAD] A question about audio file interfaces, Fons Adriaensen, (Sun Dec 1, 6:43 pm)
Re: [LAD] A question about audio file interfaces, Paul Davis, (Sat Nov 30, 11:07 pm)