[LAU] ABX test (was Re: What is the best MP3 encoder?)

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]
To: <linux-audio-user@...>
Date: Tuesday, April 2, 2013 - 8:43 pm

Quoting Ralf Mardorf :

I don't know if you're intentionally trying to misunderstand what an
ABX test or if you just do it anyway, but the test isn't to try and
trick the person doing it. In the case of testing an encoder it's
taking two identical files, only that one of them has been run through
the encoder, and see if the listener can tell which one is which. If
s/he can then the encoder isn't transparent to that person. If not, it
is.
And in that single test it doesn't matter is the person doing it is an
experienced sound engineer or a bum from behind the trash can. The
purpose is to see if the encoder is transparent for as wide an
audience as possible and the bum night happen to be sensitive to
pre-echo, smearing and similar artifacts while the engineer might not.

> The so called blind and double blind tests are already manipulated by

If you can't tell if file X is the same as file A or file B when you
get to compare them how are you going to be able to tell which one
you're listening to if you only get to hear one file, as in a real
usage situation?

The test is a challenge, and it seems it's one you for some reason
don't want to take yourself from the way you're bashing it. However
it's not a challenge where everyone is going to laugh at you if you
fail; the people making the encoder will thank you for confirming that
their work is good enough for one more person.

As for the statistical verification of the test it's obviously not all
black and white.
If you're correct in 50% or less of the test cases it means you're not
hearing any difference, are basically just guessing and that the
encoder is transparent to you. If you're correct in 75% it's a bit
trickier; you're probably hearing some differences and that means the
encoder isn't entirely transparent. If you're 90% correct the encoder
isn't transparent to your ears.

But again; it doesn't matter if you're 100, 82, 75, 67 or 50 percent
correct. Given enough test data the people making the encoder will
have learned something about their work and you have, to some extent,
learned something about your ability to hear the difference between
the original and their encoded audio.

- Peder
_______________________________________________
Linux-audio-user mailing list
Linux-audio-user@lists.linuxaudio.org
http://lists.linuxaudio.org/listinfo/linux-audio-user

Previous message: [thread] [date] [author]
Next message: [thread] [date] [author]

Messages in current thread:
[LAU] What is the best MP3 encoder?, Jostein Chr. Andersen, (Thu Mar 28, 3:10 pm)
Re: [LAU] What is the best MP3 encoder?, Marius Jan├čen, (Thu Mar 28, 4:14 pm)
Re: [LAU] What is the best MP3 encoder?, Brett McCoy, (Thu Mar 28, 3:24 pm)
Re: [LAU] What is the best MP3 encoder?, Jostein Chr. Andersen, (Thu Mar 28, 3:35 pm)
Re: [LAU] What is the best MP3 encoder?, Brett McCoy, (Thu Mar 28, 4:01 pm)
Re: [LAU] What is the best MP3 encoder?, Jostein Chr. Andersen, (Thu Mar 28, 4:08 pm)
Re: [LAU] What is the best MP3 encoder?, Raffaele Morelli, (Thu Mar 28, 3:43 pm)
Re: [LAU] What is the best MP3 encoder?, Jostein Chr. Andersen, (Thu Mar 28, 4:05 pm)
Re: [LAU] What is the best MP3 encoder?, Leigh Dyer, (Fri Mar 29, 7:27 am)
Re: [LAU] What is the best MP3 encoder?, ben, (Fri Mar 29, 6:08 pm)
Re: [LAU] What is the best MP3 encoder?, Jostein Chr. Andersen, (Fri Mar 29, 9:49 am)
Re: [LAU] What is the best MP3 encoder?, Peder Hedlund, (Tue Apr 2, 7:31 am)
Re: [LAU] What is the best MP3 encoder?, Jostein Chr. Andersen, (Tue Apr 2, 9:07 am)
Re: [LAU] What is the best MP3 encoder?, Paul Davis, (Tue Apr 2, 11:38 am)
Re: [LAU] What is the best MP3 encoder?, Jostein Chr. Andersen, (Tue Apr 2, 11:56 am)
Re: [LAU] What is the best MP3 encoder?, Peder Hedlund, (Tue Apr 2, 12:12 pm)
Re: [LAU] What is the best MP3 encoder?, Jostein Chr. Andersen, (Tue Apr 2, 12:39 pm)
Re: [LAU] What is the best MP3 encoder?, Paul Davis, (Tue Apr 2, 12:08 pm)
Re: [LAU] What is the best MP3 encoder?, Felix Homann, (Tue Apr 2, 12:56 pm)
Re: [LAU] What is the best MP3 encoder?, Paul Davis, (Tue Apr 2, 2:16 pm)
Re: [LAU] What is the best MP3 encoder?, Jostein Chr. Andersen, (Tue Apr 2, 3:41 pm)
Re: [LAU] What is the best MP3 encoder?, Fons Adriaensen, (Tue Apr 2, 12:51 pm)
Re: [LAU] What is the best MP3 encoder?, Ralf Mardorf, (Tue Apr 2, 1:02 pm)
Re: [LAU] What is the best MP3 encoder?, Ralf Mardorf, (Tue Apr 2, 12:30 pm)
Re: [LAU] What is the best MP3 encoder?, Peder Hedlund, (Tue Apr 2, 9:51 am)
Re: [LAU] What is the best MP3 encoder?, Ralf Mardorf, (Tue Apr 2, 10:36 am)
Re: [LAU] What is the best MP3 encoder?, Peder Hedlund, (Tue Apr 2, 10:59 am)
Re: [LAU] What is the best MP3 encoder?, Ralf Mardorf, (Tue Apr 2, 12:08 pm)
Re: [LAU] What is the best MP3 encoder?, Peder Hedlund, (Tue Apr 2, 12:35 pm)
Re: [LAU] What is the best MP3 encoder?, Chris Bannister, (Tue Apr 2, 3:00 pm)
Re: [LAU] What is the best MP3 encoder?, Ralf Mardorf, (Tue Apr 2, 3:24 pm)
[LAU] ABX test (was Re: What is the best MP3 encoder?), Peder Hedlund, (Tue Apr 2, 8:43 pm)
Re: [LAU] ABX test (was Re: What is the best MP3 encoder?), Florian Paul Schmidt, (Wed Apr 3, 7:12 am)
Re: [LAU] ABX test (was Re: What is the best MP3 encoder?), Raffaele Morelli, (Thu Apr 4, 5:39 am)
Re: [LAU] ABX test (was Re: What is the best MP3 encoder?), Florian Paul Schmidt, (Wed Apr 3, 6:22 pm)
Re: [LAU] ABX test (was Re: What is the best MP3 encoder?), Monty Montgomery, (Wed Apr 3, 6:31 pm)
Re: [LAU] ABX test (was Re: What is the best MP3 encoder?), Monty Montgomery, (Wed Apr 3, 9:38 pm)
Re: [LAU] ABX test (was Re: What is the best MP3 encoder?), Monty Montgomery, (Wed Apr 3, 6:49 pm)
Re: [LAU] ABX test (was Re: What is the best MP3 encoder?), Raffaele Morelli, (Thu Apr 4, 5:44 am)
Re: [LAU] ABX test (was Re: What is the best MP3 encoder?), Monty Montgomery, (Thu Apr 4, 5:48 am)
Re: [LAU] ABX test (was Re: What is the best MP3 encoder?), Monty Montgomery, (Thu Apr 4, 5:53 pm)
Re: [LAU] ABX test (was Re: What is the best MP3 encoder?), Raffaele Morelli, (Thu Apr 4, 10:02 am)
Re: [LAU] ABX test (was Re: What is the best MP3 encoder?), Florian Paul Schmidt, (Wed Apr 3, 7:30 am)
Re: [LAU] ABX test (was Re: What is the best MP3 encoder?), Florian Paul Schmidt, (Wed Apr 3, 6:43 am)
Re: [LAU] ABX test (was Re: What is the best MP3 encoder?), Raffaele Morelli, (Wed Apr 3, 7:03 am)
Re: [LAU] ABX test (was Re: What is the best MP3 encoder?), Raffaele Morelli, (Wed Apr 3, 6:18 am)
Re: [LAU] ABX test (was Re: What is the best MP3 encoder?), Raffaele Morelli, (Wed Apr 3, 9:02 am)
Re: [LAU] ABX test (was Re: What is the best MP3 encoder?), Raffaele Morelli, (Wed Apr 3, 10:35 am)
Re: [LAU] ABX test (was Re: What is the best MP3 encoder?), Florian Paul Schmidt, (Wed Apr 3, 10:51 am)
Re: [LAU] What is the best MP3 encoder?, Florian Paul Schmidt, (Tue Apr 2, 1:29 pm)
Re: [LAU] What is the best MP3 encoder?, Ralf Mardorf, (Tue Apr 2, 2:07 pm)
Re: [LAU] What is the best MP3 encoder?, Florian Paul Schmidt, (Tue Apr 2, 12:58 pm)
Re: [LAU] What is the best MP3 encoder?, Peder Hedlund, (Tue Apr 2, 1:11 pm)
Re: [LAU] What is the best MP3 encoder?, Ralf Mardorf, (Tue Apr 2, 1:42 pm)
Re: [LAU] What is the best MP3 encoder?, Raffaele Morelli, (Tue Apr 2, 12:51 pm)
Re: [LAU] What is the best MP3 encoder?, Ralf Mardorf, (Tue Apr 2, 12:44 pm)
Re: [LAU] What is the best MP3 encoder?, Gene Heskett, (Tue Apr 2, 3:32 pm)
Re: [LAU] What is the best MP3 encoder?, Ralf Mardorf, (Tue Apr 2, 4:13 pm)
Re: [LAU] What is the best MP3 encoder?, Gene Heskett, (Tue Apr 2, 5:13 pm)
Re: [LAU] What is the best MP3 encoder?, Paul Davis, (Tue Apr 2, 3:37 pm)
Re: [LAU] What is the best MP3 encoder?, Fons Adriaensen, (Tue Apr 2, 4:48 pm)
Re: [LAU] What is the best MP3 encoder?, Peder Hedlund, (Tue Apr 2, 1:34 pm)
Re: [LAU] What is the best MP3 encoder?, Ralf Mardorf, (Tue Apr 2, 2:33 pm)
Re: [LAU] What is the best MP3 encoder?, Ralf Mardorf, (Tue Apr 2, 2:36 pm)
Re: [LAU] What is the best MP3 encoder?, Bill Gribble, (Tue Apr 2, 1:28 pm)
Re: [LAU] What is the best MP3 encoder?, Raffaele Morelli, (Tue Apr 2, 9:42 am)
Re: [LAU] What is the best MP3 encoder?, Jostein Chr. Andersen, (Tue Apr 2, 9:51 am)
Re: [LAU] What is the best MP3 encoder?, Raffaele Morelli, (Tue Apr 2, 10:13 am)
Re: [LAU] What is the best MP3 encoder?, Ralf Mardorf, (Tue Apr 2, 10:56 am)
Re: [LAU] What is the best MP3 encoder?, Raffaele Morelli, (Thu Mar 28, 4:48 pm)
Re: [LAU] What is the best MP3 encoder?, Jostein Chr. Andersen, (Thu Mar 28, 5:31 pm)
Re: [LAU] What is the best MP3 encoder?, Chris Bannister, (Fri Mar 29, 2:32 am)
Re: [LAU] What is the best MP3 encoder?, Jostein Chr. Andersen, (Fri Mar 29, 9:43 am)
Re: [LAU] What is the best MP3 encoder?, Paul Davis, (Fri Mar 29, 2:43 am)
Re: [LAU] What is the best MP3 encoder? - 2, Jostein Chr. Andersen, (Thu Mar 28, 3:38 pm)
Re: [LAU] What is the best MP3 encoder? - 2, Len Ovens, (Thu Mar 28, 4:59 pm)