Mark Palko writes:
You lose information when you go from a vector to a scalar.
But what about this trick, which they told me about in high school? Combine two dimensions into one by interleaving the decimals. For example, if a=.11111 and b=.22222, then (a,b) = .1212121212.
"One way to tell an engineer from mathematician is how much they care about units."
Sure, you can do that, but there's no *continuous* mapping from a vector of real-valued elements to a scalar, so the trick still drops a useful property.
You would have to bring that up.
Actually, there are continuous mappings from scalars to vectors. See: http://en.wikipedia.org/wiki/Space-filling_curve
You can think of your entire hard drive as one massive number, if you want.
It's all in the encoding. You can both lose and gain (implicit) information depending on how you do it.
The major issue here that limits your conversion is that the scalar operators of your converted number do not give the expected results (ex: .12 + .19 != .1,.2 + .1,.9). If you have to redefine all your operators, then that would really stretch the definition of what a scalar is.
An interval on the number line has a one-to-one correspondence to any "continuous" region in any-dimensional Euclidean space. However, as Corey said, it is impossible to make the map itself continuous.
Everyone, I don't think Andrew was actually suggesting this as a useful approach to anything! He's just pointing out that it's not necessarily true that you lose information when you go from a vector to a scalar, depending on how you do it. You lose all kinds of other things (such as easy interpretability) but not necessarily information.
I would say that you have specified a method for *encoding* a vector. A scalar should have the properties that are associated to a scalar so for example, you should be able to multiply it by a vector and change the length of the vector, or you should be able to take its square root or add subtract or divide it.
It's important to distinguish between the object and the representation of the object in the computer. the vector (.1111… , .22222…) doesn't lose its vector status when you encode it as .121212… unless in fact you then treat the number .121212 as if it were just that only a single number.
There's an element of karma here. I always took sadistic pleasure from blowing students' minds with paradoxes and oddities (including this one). I should have known it would come back to haunt me.
Given the context of my original post, I'm pretty sure Phil was right and Andrew was pointing out that there are exceptions, not claiming that those exceptions would be useful for the problems I mentioned (like assigning a metric to happiness).
What about signs?
+.1111 and +.22222 gets mapped to the same scalar as -.11111 and -.22222, right? Also the ditto for +- and -+
If you code a vector into a scalar, you need the scalar plus the information of how the scalar codes for a vector to get access to the information contained in the latter. In that sence, the scalar alone does contain less information than the vector.
RE to Clay:
You could use the first digits of the scalars to code for the sign of the vector components. (e.g. 0 for + and 1 for +).
You could even reserve the first N digit of a scalar to code how the scalar should be decoded… For instance 10 zero's: what follows is just a scalar, 9 zero's and a one: what follows are the components of a two-vector and the components are both positive, etc…
In any case, you need the additional information that the scalar is used as a code for something. Then it can code for any element from any set with at most the cardinality of the continuum.
Being invertible versus assumption laden
Although invertible transformations lose no information, the other “no loss of information by assumption” was my bigger peeve.
For instance, if you assume observations are drawn from a Normal distribution – there is no loss of information from the observations given just the mean and variance. But you cannot check your assumptions with just this and worse still later you can’t change your assumptions without incurring a loss of information. So for this reason I once claimed that sufficiency should be banned from the statistical literature or at least labelled as just an annoyance. (More subtlety though, it may be useful to better define double us of data concerns.)
Something else that is intriguing though, is that in Bayesian analysis for a vector of parameters, one can focus on one any element or (measureable?) function and get a completely accurate picture of the factorization into prior and likelihood (just for that element) but not necessarily all posterior estimates one might be interested in. Not sure if there is a name for these things, but I likely need to start with O'Hagan 1976 Biometrika paper …
K?
There is an interpretation under which the loss of information claim is technically correct: if "vectors" and "scalars" are to be understood as vector spaces, and if "you lose information when going from A to B" means "there is no isomorphism from A onto B" (as opposed to "there's no one-to-one mapping between A and B"). You cannot represent a vector space with a lower-dimensional vector space without loss of information.