Re: [LAD] A question about audio file interfaces

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: Fons Adriaensen <fons@...>
Cc: Linux Audio Developers <linux-audio-dev@...>
Date: Sunday, December 1, 2013 - 4:58 pm

--047d7b6251b0d1555904ec7bf613
Content-Type: text/plain; charset=ISO-8859-1

On Sun, Dec 1, 2013 at 11:49 AM, Fons Adriaensen wrote:

> On Sun, Dec 01, 2013 at 09:08:37AM -0500, Paul Davis wrote:

basically, all files are kept open all the time. in reality, because some
systems impose limits on the number of open files, we keep a LRU cache for
file descriptors. most users never will run into a situation where the
cache is actively used. we read ahead by an amount decided by the user
(default is 5 seconds). we write in chunks of 256kB because very old
experiments suggested that on ext3 we got the best throughput with that
(this predated our use of libsndfile, however.

> Suppose you have a timeline like ABCACDBABCCA where each char is a

ardour does not attempt to optimize this. it will call read() with the file
descriptor corresponding to A 4 times, the one corresponding to B 3 times,
the one corresponding to C 4 times and the one corresponding to D once.
each time, it will read the same data each time. this is what i mean by
relying on the kernel's own buffering. those second reads typically get an
estimated bandwidth on the order of GBytes/second (i.e. basically just
memcpy from the kernel buffer cache into user space, because the kernel
already has the data cached). you could construct arbitrary scenarios where
the kernel's buffering strategy is wrong, but i'm not interested in the
work involved in designing a better strategy. there were papers i read in
the early 2000's from the late 1990's that outlined such designs (and
filesystems for audio) - AFAICT, all of this work has basically been made
irrelevant by contemporary hardware and OS performance. when you can stream
100 32 bit float mono interleaved tracks from a single (cheap) SATA disk
without real issues, i frankly don't see the point in design work to
squeeze a few more tracks out of the same setup.

--047d7b6251b0d1555904ec7bf613
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Sun, Dec 1, 2013 at 11:49 AM, Fons Adriaensen =
<fons@linuxaudi=
o.org
> wrote:
On Sun, Dec 01, 2013 at 09=
:08:37AM -0500, Paul Davis wrote:

> in ardour, the purpose of the disk i/o thread (the "butler")=
is

ut
the
s,
job.

But surely ardour does look ahead in the timeline, to open()/seek()/r=
ead()
a new file a few seconds before the data is actually required ? Or does
it keep all files open all the time ?b=
asically, all files are kept open all the time. in reality, because some sy=
stems impose limits on the number of open files, we keep a LRU cache for fi=
le descriptors. most users never will run into a situation where the cache =
is actively used. we read ahead by an amount decided by the user (default i=
s 5 seconds). we write in chunks of 256kB because very old experiments sugg=
ested that on ext3 we got the best throughput with that (this predated our =
use of libsndfile, however.
=A0
Suppose you have a timeline like ABCACDBABCCA where each char is a
very short region (maybe just a fraction of a second) and the actual
char refers to the file that region is read from. For example in 'CC&#3=
9;
near the end the first C could be a few seconds of music ending in
silence and the second C a repeat of 400 ms of the end of the first
(e.g to remove a cough or the sound of a score page being turned),
while both are just a few seconds away from the earlier occurences
of C. Would each of the regions be handled independent of any others ?<=
/blockquote>ardour does not attempt to optimize this. i=
t will call read() with the file descriptor corresponding to A 4 times, the=
one corresponding to B 3 times, the one corresponding to C 4 times and the=
one corresponding to D once. each time, it will read the same data each ti=
me. this is what i mean by relying on the kernel's own buffering. those=
second reads typically get an estimated bandwidth on the order of GBytes/s=
econd (i.e. basically just memcpy from the kernel buffer cache into user sp=
ace, because the kernel already has the data cached). you could construct a=
rbitrary scenarios where the kernel's buffering strategy is wrong, but =
i'm not interested in the work involved in designing a better strategy.=
there were papers i read in the early 2000's from the late 1990's =
that outlined such designs (and filesystems for audio) - AFAICT, all of thi=
s work has basically been made irrelevant by contemporary hardware and OS p=
erformance. when you can stream 100 32 bit float mono interleaved tracks fr=
om a single (cheap) SATA disk without real issues, i frankly don't see =
the point in design work to squeeze a few more tracks out of the same setup=
.
=A0

--047d7b6251b0d1555904ec7bf613--

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
[LAD] A question about audio file interfaces, Fons Adriaensen, (Sat Nov 30, 10:57 pm)
Re: [LAD] A question about audio file interfaces, Fred Gleason, (Sun Dec 1, 3:42 am)
Re: [LAD] A question about audio file interfaces, Erik de Castro Lopo, (Sun Dec 1, 1:43 am)
Re: [LAD] A question about audio file interfaces, Devin Anderson, (Sun Dec 1, 12:32 am)
Re: [LAD] A question about audio file interfaces, Fons Adriaensen, (Sun Dec 1, 11:48 am)
Re: [LAD] A question about audio file interfaces, Paul Davis, (Sun Dec 1, 2:08 pm)
Re: [LAD] A question about audio file interfaces, Fons Adriaensen, (Sun Dec 1, 4:49 pm)
Re: [LAD] A question about audio file interfaces, Paul Davis, (Sun Dec 1, 4:58 pm)
Re: [LAD] A question about audio file interfaces, Fons Adriaensen, (Sun Dec 1, 6:43 pm)
Re: [LAD] A question about audio file interfaces, Paul Davis, (Sat Nov 30, 11:07 pm)