how to find the best ADC step size?

CASTalk.com
Discussion of DSP, FPGA, storage and embedded system.

FAQ

Memberlist

Profile

how to find the best ADC step size? Goto page 1, 2 Next

CASTalk.com Forum Index -> DSP

Author

Message

lucy
Guest

Posted: Wed Dec 22, 2004 4:41 am Post subject: how to find the best ADC step size?

Hi all, Suppose my input data follows Gaussian distribution with mean u and variance sigma^2. If I want to design 6-bit ADC... what should be the optimal step size for this ADC? Thanks a lot -L

Scott Seidman
Guest

Posted: Wed Dec 22, 2004 5:02 am Post subject: Re: how to find the best ADC step size?

"lucy" <losemind@yahoo.com> wrote in
news:cqacbp$bbc$1@news.Stanford.EDU:

Quote:

Hi all,

Suppose my input data follows Gaussian distribution with mean u and
variance sigma^2.

If I want to design 6-bit ADC... what should be the optimal step size
for this ADC?

Thanks a lot

-L

If ADC is Analog to Digital Converter, the distribution of your data has
nothing to do with the resolution. The range of your data does, though.
You can sacrifice range for resolution, and resolution for range.

Why 6 bits?

Scott

glen herrmannsfeldt
Guest

Posted: Wed Dec 22, 2004 5:22 am Post subject: Re: how to find the best ADC step size?

Scott Seidman wrote:

Quote:

"lucy" <losemind@yahoo.com> wrote in
news:cqacbp$bbc$1@news.Stanford.EDU:

Suppose my input data follows Gaussian distribution with mean u and
variance sigma^2.

If I want to design 6-bit ADC... what should be the optimal step size
for this ADC?

If ADC is Analog to Digital Converter, the distribution of your data has
nothing to do with the resolution. The range of your data does, though.
You can sacrifice range for resolution, and resolution for range.

Well, Gaussian distributions have infinite range, though with
ever decreasing probability. This does sound like a homework
problem, and possibly some important information was left out.

What parameter is being optimized through step size choice?
I don't believe there is only one answer to this problem.

-- glen

Scott Seidman
Guest

Posted: Wed Dec 22, 2004 5:34 am Post subject: Re: how to find the best ADC step size?

glen herrmannsfeldt <gah@ugcs.caltech.edu> wrote in
news:cqaejd$4gl$1@gnus01.u.washington.edu:

Quote:

Scott Seidman wrote:
"lucy" <losemind@yahoo.com> wrote in
news:cqacbp$bbc$1@news.Stanford.EDU:

Suppose my input data follows Gaussian distribution with mean u and
variance sigma^2.

If I want to design 6-bit ADC... what should be the optimal step size
for this ADC?

If ADC is Analog to Digital Converter, the distribution of your data
has nothing to do with the resolution. The range of your data does,
though. You can sacrifice range for resolution, and resolution for
range.

Well, Gaussian distributions have infinite range, though with
ever decreasing probability. This does sound like a homework
problem, and possibly some important information was left out.

What parameter is being optimized through step size choice?
I don't believe there is only one answer to this problem.

-- glen

Lucy isn't a student, but she does hold something near a record for thread
initiations in cssm. Frankly, she tends to use this newsgroup instead of
hauling out a textbook, or even doing a google search to try to find what
she needs to know. If I had noticed it was her post, I probably would have
skipped it.

Scott

lucy
Guest

Posted: Wed Dec 22, 2004 6:45 am Post subject: Re: how to find the best ADC step size?

"Scott Seidman" <namdiesttocs@mindspring.com> wrote in message
news:Xns95C6C72198B8Ascottseidmanmindspri@130.133.1.4...

Quote:

glen herrmannsfeldt <gah@ugcs.caltech.edu> wrote in
news:cqaejd$4gl$1@gnus01.u.washington.edu:

Scott Seidman wrote:
"lucy" <losemind@yahoo.com> wrote in
news:cqacbp$bbc$1@news.Stanford.EDU:

Suppose my input data follows Gaussian distribution with mean u and
variance sigma^2.

If I want to design 6-bit ADC... what should be the optimal step size
for this ADC?

If ADC is Analog to Digital Converter, the distribution of your data
has nothing to do with the resolution. The range of your data does,
though. You can sacrifice range for resolution, and resolution for
range.

Well, Gaussian distributions have infinite range, though with
ever decreasing probability. This does sound like a homework
problem, and possibly some important information was left out.

What parameter is being optimized through step size choice?
I don't believe there is only one answer to this problem.

-- glen

Lucy isn't a student, but she does hold something near a record for thread
initiations in cssm. Frankly, she tends to use this newsgroup instead of
hauling out a textbook, or even doing a google search to try to find what
she needs to know. If I had noticed it was her post, I probably would
have
skipped it.

Scott

Hi Scott,

I did not know that my posts were so disgusting... to you...

But I did try to ask questions that are not easily found solutions in
textbooks... some problems are in fact initiated by myself. I just like
problem-solving... It is hard to find guys interested in problem-solving
nearby... I thought newsgroup might be a better place... if you find my
problems are so trivial that a few googling + book hunting can work out,
please do let me know... please let me know on which book you can find the
solution to the above problem. I'd really like to know ... newsgroup serves
as pointers, right?

Clay S. Turner
Guest

Posted: Wed Dec 22, 2004 8:03 am Post subject: Re: how to find the best ADC step size?

Hello Lucy,

Try looking in "Digital Processing of Speech Signals" by Rabiner & Schafer.
There is a bunch of info on matching the quantization to the distribution of
a signal. Basically you are maximizing the entropy. I.e gain the most info
per bit that you can.

Clay

"lucy" <losemind@yahoo.com> wrote in message
news:cqacbp$bbc$1@news.Stanford.EDU...

Quote:

Hi all,

Suppose my input data follows Gaussian distribution with mean u and
variance sigma^2.

If I want to design 6-bit ADC... what should be the optimal step size for
this ADC?

Thanks a lot

-L

Clay S. Turner
Guest

Posted: Wed Dec 22, 2004 8:03 am Post subject: Re: how to find the best ADC step size?

"glen herrmannsfeldt" <gah@ugcs.caltech.edu> wrote in message
news:cqaejd$4gl$1@gnus01.u.washington.edu...

Quote:

Scott Seidman wrote:
"lucy" <losemind@yahoo.com> wrote in
news:cqacbp$bbc$1@news.Stanford.EDU:

Suppose my input data follows Gaussian distribution with mean u and
variance sigma^2.

If I want to design 6-bit ADC... what should be the optimal step size
for this ADC?

If ADC is Analog to Digital Converter, the distribution of your data has
nothing to do with the resolution. The range of your data does, though.
You can sacrifice range for resolution, and resolution for range.

Well, Gaussian distributions have infinite range, though with
ever decreasing probability. This does sound like a homework
problem, and possibly some important information was left out.

Hello Glen,

One way is to divide the area under the bell curve into equal area
partitions. In this case 2^6 of them. The abscissal value for each of the
partitions becomes a transition point for the quantization. This maximizes
the entropy (i.e., the information) since each quantization will be equally
represented. And the entopy is maximized when each state has equal
probability. This in not unlike Huffman's idea where his coding scheme
attempts to make the length of each symbol times its frequency of occurance
be the same for all symbols.

Clay

Quote:

What parameter is being optimized through step size choice?
I don't believe there is only one answer to this problem.

-- glen

Randy Yates
Guest

Posted: Wed Dec 22, 2004 5:30 pm Post subject: Re: how to find the best ADC step size?

"Clay S. Turner" <Physics@Bellsouth.net> writes:

Quote:

[...]
One way is to divide the area under the bell curve into equal area
partitions. In this case 2^6 of them. The abscissal value for each of the
partitions becomes a transition point for the quantization. This maximizes
the entropy (i.e., the information) since each quantization will be equally
represented. And the entopy is maximized when each state has equal
probability.

Isn't this a mapping from Gaussian to uniform? And we all know that
uniform has the greatest entropy.

Is this related to the concept of "vector quantization"?

Quote:

This in not unlike Huffman's idea where his coding scheme
attempts to make the length of each symbol times its frequency of occurance
be the same for all symbols.

I'm having trouble with this. Yes, I see that a Huffman code attempts to make
(symbol length)*(symbol frequency) constant over all symbols. How does change
anything about the underlying distribution, though?
--
Randy Yates
Sony Ericsson Mobile Communications
Research Triangle Park, NC, USA
randy.yates@sonyericsson.com, 919-472-1124

Scott Seidman
Guest

Posted: Wed Dec 22, 2004 6:59 pm Post subject: Re: how to find the best ADC step size?

"lucy" <losemind@yahoo.com> wrote in
news:cqajiv$hsm$1@news.Stanford.EDU:

Quote:

Hi Scott,

I did not know that my posts were so disgusting... to you...

But I did try to ask questions that are not easily found solutions in
textbooks... some problems are in fact initiated by myself. I just
like problem-solving... It is hard to find guys interested in
problem-solving nearby... I thought newsgroup might be a better
place... if you find my problems are so trivial that a few googling +
book hunting can work out, please do let me know... please let me
know on which book you can find the solution to the above problem. I'd
really like to know ... newsgroup serves as pointers, right?

lucy-

I don't find your posts disgusting, just somewhat tedious. The netscan
page lists your posts in cssm under your currently used email addy (and I
seem to remember at least one more address before the current one) for
the time period between 7/1/04 and 9/30/04. In that three-month period,
you initiated 77 threads, and your first use of that name was on 7/28!
That's more threads, by about a factor of 5, beyond your nearest
competitor. Then,

As many of those posts were requests for very basic information about GUI
programming in Matlab (which participants seemed to have taught you,
quite patiently and generously), my impression during that binge of
questioning was that you were fairly bright, but embarked on a project
in an unfamiliar environment that you didn't want to take the time to
learn. Then, the posts moved over to signal processing, where you
appeared to have absolutely no training, but you expected to learn this
stuff on cssm. Signal processing questions covered: basic fft
properties, basic sampling theorem, filter design, image processing, and
some other topics, every one of which is well covered in one of two
textbooks that you can find on just about every library's shelves. I'd
suggest "Signals and Systems" by Oppenheim and Willsky, "Digital Signal
Processing" by Oppenheim and Schaffer, and the lovely image processing
book that can be found on the mathworks site under textbooks. My
recommendations are somewhat dated, as I'm sure that more modern
references on signals and systems cover a better mix of analog and
digital techniques. Some of the docs for the Matlab toolboxes have fine
reference lists, and they don't put those list in just so endusers can
ignore them.

Personally, I'd recommend you consider taking a course on the topic, as
it will be a time saver in the long run. Help is help, but when you post
50 threads on one subject, consider going out and learning it right.
Also, a good rule of thumb is to consider spending an 45 minutes or an
hour trying to do something (like everybody who contributes to your
threads did at some time) before asking for help with it. Then, try
googling for it to see if you can help yourself without passing the hat.
In fact, if you've googled this particular question in usenet groups,
you'll find that you've already asked almost exactly the same question,
but for the more complex 2D case!

CSSM is a fine resource to get you over the rough spots, but its not the
participants' responsibility to provide you with on the job training in
areas you aren't proficient in.

Scott

Clay S. Turner
Guest

Posted: Wed Dec 22, 2004 7:47 pm Post subject: Re: how to find the best ADC step size?

Hello Randy,
comments below:

"Randy Yates" <randy.yates@sonyericsson.com> wrote in message
news:xxppt12cy6f.fsf@usrts005.corpusers.net...

Quote:

"Clay S. Turner" <Physics@Bellsouth.net> writes:
[...]
One way is to divide the area under the bell curve into equal area
partitions. In this case 2^6 of them. The abscissal value for each of the
partitions becomes a transition point for the quantization. This
maximizes
the entropy (i.e., the information) since each quantization will be
equally
represented. And the entopy is maximized when each state has equal
probability.

Isn't this a mapping from Gaussian to uniform? And we all know that
uniform has the greatest entropy.

Basically except for the lack of a Jacobian. And going to a form which
maximizes our entropy is the goal.

Quote:

Is this related to the concept of "vector quantization"?

Probably - but I havn't done much with this, so I'll have to guess here. But
since the concept of maximizing entropy is a wonderful way of handling
quantization without history, so I would be surprised if it hasn't been used
for quantizing vocoder vectors. Remember in all of this process, we are
assuming no sample to sample correlation. And in speech this is not quite
true.

Quote:

This in not unlike Huffman's idea where his coding scheme
attempts to make the length of each symbol times its frequency of
occurance
be the same for all symbols.

I'm having trouble with this. Yes, I see that a Huffman code attempts to
make
(symbol length)*(symbol frequency) constant over all symbols. How does
change
anything about the underlying distribution, though?

The idea is to try to make each symbol contribute equally to the overall
process. Imagine looking at your data after a huge number of symbols was
received. The idea is to make the info provided by each type of symbol
contribute equally. And in data compression, the idea is to find the minimum
size for all of the data for a fixed information content.

I hope this helps.

Clay

Quote:

--
Randy Yates
Sony Ericsson Mobile Communications
Research Triangle Park, NC, USA
randy.yates@sonyericsson.com, 919-472-1124

Randy Yates
Guest

Posted: Wed Dec 22, 2004 8:55 pm Post subject: Re: how to find the best ADC step size?

"Clay S. Turner" <Physics@Bellsouth.net> writes:

Quote:

[...]
I'm having trouble with this. Yes, I see that a Huffman code attempts to
make
(symbol length)*(symbol frequency) constant over all symbols. How does
change
anything about the underlying distribution, though?

The idea is to try to make each symbol contribute equally to the overall
process. Imagine looking at your data after a huge number of symbols was
received. The idea is to make the info provided by each type of symbol
contribute equally.

But this Huffman coding won't do that. Choosing a representation for a
symbol doesn't change the probability of the symbol occurring. It
does, however, minize the average symbol rate - I certainly see
that. Perhaps I'm being blind?
--
Randy Yates
Sony Ericsson Mobile Communications
Research Triangle Park, NC, USA
randy.yates@sonyericsson.com, 919-472-1124

glen herrmannsfeldt
Guest

Posted: Wed Dec 22, 2004 11:33 pm Post subject: Re: how to find the best ADC step size?

Someone wrote:

Quote:

I'm having trouble with this.
Yes, I see that a Huffman code attempts to make
(symbol length)*(symbol frequency) constant over
all symbols. How does change
anything about the underlying distribution, though?

"Clay S. Turner" <Physics@Bellsouth.net> writes:
The idea is to try to make each symbol contribute equally to the overall
process. Imagine looking at your data after a huge number of symbols was
received. The idea is to make the info provided by each type of symbol
contribute equally.

Randy Yates wrote:

Quote:

The gaussian has infinite tails, so it isn't possible to cover
the range with a linear scale ADC. One could then ask for what
part of the distribution, when covered with the steps of a six
bit ADC, are the symbol probabilities most equal.

Without doing the math it isn't so obvious either way.
For the case of a very large (lim --> infinity) most of
the symbols will have almost no probability. In the
limit of very small step size, and assuming that the lower
and upper step get the tails, again most steps have almost
no probability.

So, it would seem that somewhere in between the probability
would be more equal. One should then define the appropriate
function of step size and find the minimum point.

-- glen

Scott Seidman
Guest

Posted: Wed Dec 22, 2004 11:37 pm Post subject: Re: how to find the best ADC step size?

glen herrmannsfeldt <gah@ugcs.caltech.edu> wrote in news:cqceia$n9j$1
@gnus01.u.washington.edu:

Quote:

I think the question is impossible to address in a vacuum. The theoretical
handling, while possibly interesting, must take a back seat to the
engineering concerns. Start with the questions "Why am I sampling the
signal, and what will I be doing with the signal after I sample it?"

Then, you can start talking about what happens when the signal falls
outside your ADC range. You might even choose to investigate the
possibility of a non-linear ADC. I've never heard of such a thing, but
there's no reason why it can't be implemented.

Scott

Jerry Avins
Guest

Posted: Thu Dec 23, 2004 12:26 am Post subject: Re: how to find the best ADC step size?

glen herrmannsfeldt wrote:

Quote:

Someone wrote:

I'm having trouble with this.

Yes, I see that a Huffman code attempts to make

(symbol length)*(symbol frequency) constant over

all symbols. How does change

anything about the underlying distribution, though?

"Clay S. Turner" <Physics@Bellsouth.net> writes:

The idea is to try to make each symbol contribute equally to the
overall process. Imagine looking at your data after a huge number of
symbols was received. The idea is to make the info provided by each
type of symbol contribute equally.

Randy Yates wrote:

But this Huffman coding won't do that. Choosing a representation for a
symbol doesn't change the probability of the symbol occurring. It
does, however, minize the average symbol rate - I certainly see
that. Perhaps I'm being blind?

The gaussian has infinite tails, so it isn't possible to cover
the range with a linear scale ADC. One could then ask for what
part of the distribution, when covered with the steps of a six
bit ADC, are the symbol probabilities most equal.

...

For non-linear encoding, it actually is possible. One way is to divide
the cumulative distribution into six parts equal (divide the density
function into six equal areas), then have the ADC report the sextile*
that the sample falls into. Infinite tails don't bother that scheme.

Jerry
_________________________
* Rarely the kitchen floor.
--
Engineering is the art of making what you want from things you can get.
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯

Clay S. Turner
Guest

Posted: Thu Dec 23, 2004 12:37 am Post subject: Re: how to find the best ADC step size?

"Scott Seidman" <namdiesttocs@mindspring.com> wrote in message
news:Xns95C78A9EB3837scottseidmanmindspri@130.133.1.4...

Quote:

Then, you can start talking about what happens when the signal falls
outside your ADC range. You might even choose to investigate the
possibility of a non-linear ADC. I've never heard of such a thing, but
there's no reason why it can't be implemented.

Hello Scott,
Anytime you speak over a landline where the signal is carried digitally, the
speech is quantized nonlinearly. Depending on the part of the world you are
in, it will be either mu-law (North America) or A-law (pretty much the rest
of the world save for a few place like a parts of the Philliphines).

A lot of early work with nonlinear quantization was done using Laplacian and
Gamma densities as models for the speech.

Look up:

Paez, M.D., and Glisson, T.H., "Minimum Mean Squared Error Quantization in
Speech", IEEE Trans. Comm., Vol. Com-20, pp. 225-230, April 1972

and

Max, J., "Quantizing for Minimum Distortion", IRE Trans. Inform. Theory, Vol
IT-6, pp 7-12, March 1960

for more details.

Clay

Quote:

Scott

	CASTalk.com Forum Index -> DSP	All times are GMT Goto page 1, 2 Next
Page 1 of 2

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum