Linear algebra and vector spaces

[TheMotorcycleBoy]

Hi folks,

Anyone know about maths?

I'm currently reading Dancing with Qubits. The first couple of 100 pages or so, are a brush up/intro to some of the maths concepts which may be of use in later parts of the book.

Whilst I have a fairly good educational background in maths (A levels in Pure and Applied, and Further Pure and Applied, plus 2.1 BEng), I don't recall doing any linear algebra much more advanced than Group Theory, and a simple introduction to vectors and matrices. The Quant Compute book above has already introduced the extensions to Groups, i.e. rings and fields, which I can totally grasp along with application to the real line (ℝ) and real plane and complex numbers etc.

What I'm a little unclear about is the meaning of the term a "vector space over a field". At first I thought that a vector space (over a field) was merely a set of vectors (i.e. multiple dimensioned entities) whose elements originate from the elements in the field. As such I'd imagined that an example of one could be the vectors comprising the x,y coordinates of a unit circle centred at the origin, where the field is the set of real numbers.

However on looking at a wiki page, I can see that I'm probably wrong in this case, since this statement

The first four axioms mean that V is an abelian group under addition.

would imply closure under addition, which is certainly not the case, since (0,1) and (0.523,0.851) (see [1]) are both points on the circle, but their sum is (0.523, 1.851) which is a point outside the circle hence closure is not a property held by the unit circle vectors.

So I'm wondering if someone could confirm my surmisal, and perhaps come up with a good example of a vector space over a field.

thanks Matt

[1] the second coord being that where the subtended angle a radius makes against the x-axis is 45*

[Alaric]

I don't recall doing any linear algebra much more advanced than Group Theory, and a simple introduction to vectors and matrices.

If Group Theory was year 1, term 1 of a Maths degree, then Vector Spaces were term 2.

The wiki article on the subject may or may not be reliable.

https://en.wikipedia.org/wiki/Vector_space

It's a branch of mathematics in its own right with as is frequent for post A level Maths, a language all of its own.

[TheMotorcycleBoy]

Fine. Thanks, but you could please satisfy my implied requests?

So I'm wondering if someone could confirm my surmisal, and perhaps come up with a good example of a vector space over a field.

Matt

[modellingman]

Fine. Thanks, but you could please satisfy my implied requests?

So I'm wondering if someone could confirm my surmisal, and perhaps come up with a good example of a vector space over a field.

Matt

If your surmisal is that your statement about vectors drawn from the origin of the unit circle to points on the circumference is incorrect, then I can confirm it is indeed incorrect (but see below for why it is incorrect).

Alaric has suggested the Wikipedia article on Vector Spaces and this is a reasonable place to start. It defines a vector space over a field as a set of vectors V and a field F with two operations: conventionally these are use called "addition" and "multiplication". Addition is a mapping from the "space" V*V to V. As you state, this set and the operation of "addition" are required to conform to the axioms of a group. Multiplication (or scalar multiplication as it is called in the Wikipedia definition) is a mapping from F*V to V. The multiplication operation on F*V also needs to satisfy a number of group-like axioms.

There are several examples of vector spaces given in the Wikipedia article.

Your unit circle example is incorrect because, with your definition of addition, the set of vectors is, as you correctly identify, not closed. However, if you change your definition of addition, you can make it into a closed set of vectors (hint: think in terms of polar co-ordinates and clock arithmetic). With a little bit of thought, you should also be able to define this set and this new addition operation as a group. I'll leave it to you to figure out whether there is a corresponding field, F and a multiplication operation that makes your vector group into a vector space over the field.

For vector spaces (and indeed other algebraic structures such as groups, rings and fields) it is important to bear in mind that the words "addition" and "multiplication" are simply shorthand for mathematical operations. Provided the operations satisfy the axioms of the structure, they do not have to bear much (or even any) resemblance to what is regarded as "addition" and "multiplication" in the everyday world of arithmetic. As in all mathematics, being clear about the definitions of the entities your are working with is vital.

[Spet0789]

As an aside, how did you manage to do an Engineering degree without a lot of vectors and matrix algebra?

[TheMotorcycleBoy]

As an aside, how did you manage to do an Engineering degree without a lot of vectors and matrix algebra?

We did some vectors and matrix algebra. I wouldn't say we did "a lot" mind you. We certainly did not define abstractions such as "vector spaces over fields", in fact groups weren't mentioned in my degree. I learnt about groups (but not rings or fields) the year before (1992!) in my Further Maths A level.

My B'Eng was incredibly broad in the first year, and there presumably wasn't the time to do a massive amount of linear algebra. In the final year, I tended to specialise in control systems, signal processing (especially DSP) but mainly in software engineering.

I've had some thoughts re. MMs post and have done some background reading. Will try to post back later today or tomorrow. I must get on with the day job right now

Matt

[TheMotorcycleBoy]

If your surmisal is that your statement about vectors drawn from the origin of the unit circle to points on the circumference is incorrect, then I can confirm it is indeed incorrect (but see below for why it is incorrect).
....

Indeed it was - thanks

Your unit circle example is incorrect because, with your definition of addition, the set of vectors is, as you correctly identify, not closed. However, if you change your definition of addition, you can make it into a closed set of vectors (hint: think in terms of polar co-ordinates and clock arithmetic). With a little bit of thought, you should also be able to define this set and this new addition operation as a group. I'll leave it to you to figure out whether there is a corresponding field, F and a multiplication operation that makes your vector group into a vector space over the field.
...

I'm now a tiny bit clearer on this. I often get most puzzled on the exact contextual meaning of the terminology. What foxed me was/is the "over a field" bit. I now *think* that this merely means (in terms of the vector space) that (the field) is where the scalars come from.

Anyway I had go at the unit circle problem. You'll probably laugh at my attempt, I'm being a bit generous with the operation definitions. Here goes:

Let V be the vector space of the cartesian coords of a unit circle centered at (0,0) over a field F. Let the field F be R i.e. the real numbers.

Let v be an element in the space such that v = (v1, v2) = (cos(p), sin(p)) where p is the angle in radians between a line from (0,0) to (v1,v2) and the +ve x axis, in a counterclockwise direction.

u is another different element in the space where u = (u1, u2) = (cos(q), sin(q)).

I defined + such that

v + u = (v1 + u1, v2 + u2)

where

v1 + u1 = cos(arccos(v1) + arccos(u1)) and v2 + u2 = sin(arcsin(v1) + arcsin(u1)).

If k is a scalar i.e. an element in the field F then I define scalar multiplication to be such that:

kv = (cos(k * arccos(v1)), sin(k * arcsin(v2)) where * is "regular" multiplication.

Matt

[modellingman]

Matt

Broadly, you have got it, so well done.

I would add several things:

First, if you work in polar co-ordinates so that rather than using (x,y) as Cartesian co-ordinates you use (r,θ), where r is the straight-line distance of a point from the origin [(0,0) in Cartesian co-ordinates] and θ is the angle between the x-axis and the line formed by joining the origin to the point, then life becomes much simpler. I'll use [θ] to denote the vector from the origin to the point expressed as (1,θ) in polar co-ordinates and I'll restrict θ to 0<=θ<2∏. You should recognise the set V, defined as V= {[θ]: 0<=θ<2∏}, as the set of your unit circle vectors.

Define "addition" as a binary operator on V (denoting it by the symbol "⨁") as:

[θ] ⨁ [φ] = [θ+φ(mod 2∏)]

Set V and operation ⨁ satisfy the axioms of an Abelian group: the set is closed under the operation, the operation is both commutative and associative, the identity is [0] and every member of the set has an inverse under the operation. [The inverse of [θ] (0<θ<2∏) is [2∏-θ], whilst the inverse of [0] is itself.]

Second, taking the field F as the set of real numbers (with conventional + and × as the 2 field operations) defining a scalar multiplication operation ⨂ as:

f ⨂ [θ]= [f×θ(mod 2∏)]

all that's now needed is demonstrate that the remaining axioms for a vector space V over field F are satisfied. (They are.)

Your Cartesian co-ordinate version (and the resulting operation definitions with cos's and arcos's, etc stuffed together) implements the same ideas and is not wrong, though it does gloss over a few of the finer details and you definitely have an engineer's approach to mathematical rigour - ie close to non-existent - which I guess is why you chose to study Engineering rather than Mathematics for your degree. No offence intended!

[servodude]

In the final year, I tended to specialise in control systems, signal processing (especially DSP) but mainly in software engineering.

In that case you'll need practical language without this "abstract maths" faffing about

A system response at any given frequency has a gain and phase (or angle and magnitude, quantum and direction) i.e. a vector
- over a frequency range this gives you a vector space

Then think Nyquist diagram vs Bode plot vs Nichols chart (i.e different ways of illustrating these) as fields with different rules for manipulation, addition and multiplication (changing gain)

sort of

- sd

[TheMotorcycleBoy]

Matt

Broadly, you have got it, so well done.

I would add several things:

First, if you work in polar co-ordinates so that rather than using (x,y) as Cartesian co-ordinates you use (r,θ), where r is the straight-line distance of a point from the origin [(0,0) in Cartesian co-ordinates] and θ is the angle between the x-axis and the line formed by joining the origin to the point, then life becomes much simpler. I'll use [θ] to denote the vector from the origin to the point expressed as (1,θ) in polar co-ordinates and I'll restrict θ to 0<=θ<2∏. You should recognise the set V, defined as V= {[θ]: 0<=θ<2∏}, as the set of your unit circle vectors.

I had already considered the polar coordinate option, however I discounted it, since as you've stated above it reduces to a single dimension since for all r, r=1. I accept that it would still have comprised a "vector space over a field", but I wanted a slightly more chewy example.

Define "addition" as a binary operator on V (denoting it by the symbol "⨁") as:

[θ] ⨁ [φ] = [θ+φ(mod 2∏)]

Set V and operation ⨁ satisfy the axioms of an Abelian group: the set is closed under the operation, the operation is both commutative and associative, the identity is [0] and every member of the set has an inverse under the operation. [The inverse of [θ] (0<θ<2∏) is [2∏-θ], whilst the inverse of [0] is itself.]

Second, taking the field F as the set of real numbers (with conventional + and × as the 2 field operations) defining a scalar multiplication operation ⨂ as:

f ⨂ [θ]= [f×θ(mod 2∏)]

all that's now needed is demonstrate that the remaining axioms for a vector space V over field F are satisfied. (They are.)

Sure. Tell me where do you get those symbols from? I use Ubuntu 20.04 wondered if you used anything similar.

you definitely have an engineer's approach to mathematical rigour - ie close to non-existent - which I guess is why you chose to study Engineering rather than Mathematics for your degree. No offence intended!

Ha ha! None taken my friend I do have a pretty intense day job you know! Was at it from 4.20am till 5.00pm yesterday....and even had to drive in (60 mile round trip) to finally collect some books and boards I'd left there back in March, plus grab some stationery, the spare pair of glasses I'd left there etc. etc.

Re. Engineering not Maths, whilst scoring very highly in Maths A level (grade A and grade C in further), and loving Maths, I'm fascinated by the applications of it. I was at the time, also busy pulling apart motorbikes, engines, rewiring stuff, making signal generators and stuff. I flirted at first with doing AstroPhysics but the best uni (St Andrews) was too far from my Bedfordshire roots, plus I doubted it's use as a marketable skill, since there were few installed radio telescopes advertising junior/trainee positions at the time!

Engineering (E&E) was the logical choice, and for me, definitely the best re. Spet0789's earlier remark, we were far more into solving DEs, contour integrals, div, curl, grad etc. However, ultimately it was software which interested me the most, despite back in 1990 in my first year absolutely hating computers, and thinking they were for nerds. Wow Uni really transformed me.

later Matt

[modellingman]

Tell me where do you get those symbols from? I use Ubuntu 20.04 wondered if you used anything similar.

Nothing more exotic than MSOffice, I'm afraid. Theta, phi and pi came from the standard Insert character command onto a cell in Excel. The circled operators from the Equation Editor. Copy/paste took care of the rest. I confess I was slightly surprised that they all transferred into the LF browser input field without fuss.

[TheMotorcycleBoy]

Tell me where do you get those symbols from? I use Ubuntu 20.04 wondered if you used anything similar.

Nothing more exotic than MSOffice, I'm afraid. Theta, phi and pi came from the standard Insert character command onto a cell in Excel. The circled operators from the Equation Editor. Copy/paste took care of the rest. I confess I was slightly surprised that they all transferred into the LF browser input field without fuss.

Hi MM,

On the subject of vectors themselves, my book keeps mentioning

vT i.e. A transpose of a vector. I didn't there was such a thing, I thought it was something that only square matrices had.

Anyway, is the act of transposing a vector merely turning a row vector into a column vector and vice-versa?

Thanks Matt

[TheMotorcycleBoy]

Yes. This clip cleared it up.

https://www.khanacademy.org/math/linear ... f-a-vector

[9873210]

I had already considered the polar coordinate option, however I discounted it, since as you've stated above it reduces to a single dimension since for all r, r=1. I accept that it would still have comprised a "vector space over a field", but I wanted a slightly more chewy example.

They are the same one dimensional vector space in either polar or cartesian coordinates. The representation does not change the underlying structure.
The only way one is more chewy than the other is it makes you work harder.

Since nobody's said it: There is no such thing as "a vector space not over a field". A "vector space over a field" and a "vector space" are the same thing. Good old three dimensional cartesian vectors, which engineers use all the time, are a fine example of a vector space over a field.

[TheMotorcycleBoy]

I had already considered the polar coordinate option, however I discounted it, since as you've stated above it reduces to a single dimension since for all r, r=1. I accept that it would still have comprised a "vector space over a field", but I wanted a slightly more chewy example.

They are the same one dimensional vector space in either polar or cartesian coordinates. The representation does not change the underlying structure.
The only way one is more chewy than the other is it makes you work harder.

Since nobody's said it: There is no such thing as "a vector space not over a field". A "vector space over a field" and a "vector space" are the same thing. Good old three dimensional cartesian vectors, which engineers use all the time, are a fine example of a vector space over a field.

Sure.

But I wanted a non-trivial scenario, as you say, to make me work harder!

[SalvorHardin]

A bit late. Dredging up Maths and Physics from 25 years ago, and this is right on my limit (I've been reading my notes from then; can't follow the Maths but can still follow the Physics (just)).

The Minkowski four-dimensional spacetime of Special Relativity is an example of a vector space over a field.

The Reimann four-dimensional spacetime of General Relativity is a Tensor space over a field (vectors are simple tensors). Some of us will have seen this represented in TV programmes about relativity with ball bearings rolling on rubber sheets where depressions represent gravity wells.

Highly impractical I know. I'm currently struggling with Group Theory in a third year Pure Maths course, having not touched the subject for 25 years, knowing that my younger self would waltz through it

[9873210]

real
I had already considered the polar coordinate option, however I discounted it, since as you've stated above it reduces to a single dimension since for all r, r=1. I accept that it would still have comprised a "vector space over a field", but I wanted a slightly more chewy example.

They are the same one dimensional vector space in either polar or cartesian coordinates. The representation does not change the underlying structure.
The only way one is more chewy than the other is it makes you work harder.

Since nobody's said it: There is no such thing as "a vector space not over a field". A "vector space over a field" and a "vector space" are the same thing. Good old three dimensional cartesian vectors, which engineers use all the time, are a fine example of a vector space over a field.

Sure.

But I wanted a non-trivial scenario, as you say, to make me work harder!

It makes you work harder at trigonometry, not linear algebra. Which makes it harder to learn linear algebra.

With respect to vector spaces, "dimension" is a term of art. You are using it in a slightly different sense, which caused my comment.

One of the key theorems for vector spaces is that each vector space has a (non-unique) basis and a (unique) dimension.
A basis is the smallest set of vectors that can be in linearly combined to give any vector in the space. The size of any basis is the dimension of the vector space.

In the vector space of rotations any single non-zero rotation is a basis (multiply by the appropriate real number to get any rotation). So the dimension of the space is one. This is true whether you use cartesian or polar coordinates.

[TheMotorcycleBoy]

I had already considered the polar coordinate option, however I discounted it, since as you've stated above it reduces to a single dimension since for all r, r=1. I accept that it would still have comprised a "vector space over a field", but I wanted a slightly more chewy example.

They are the same one dimensional vector space in either polar or cartesian coordinates. The representation does not change the underlying structure.
The only way one is more chewy than the other is it makes you work harder.

Since nobody's said it: There is no such thing as "a vector space not over a field". A "vector space over a field" and a "vector space" are the same thing. Good old three dimensional cartesian vectors, which engineers use all the time, are a fine example of a vector space over a field.

I had another read of Dancing with Qubits last night:

Let F be a field, for example R or C, and V be a set of objects. These objects are called vectors and are shown in bold as v. We are interested in defining a special kind of multiplication, called scalar multiplication, and addition.

If s in F then we insist sv is in V for all v in V. This means the set V is closed under multiplication by scalars from the field F. While V may have some kind of multiplication defined between its elements, we do not consider it here.

For any v1 and v2 in V, we also insist v1 + v2 is in V and that the addition is commutative. The V is closed under addition. In fact, we demand V has an element O and additive inverses so that V is an commutative additive group.

V is almost a vector over space over F but we have to insist on a few more conditions related to scalar multiplication. They concern the usual arithmetic properties we first saw with numbers. Let s1 and s2 be in F. All the following must hold:

1v = v for 1, the multiplicative identity of F
s1(v1 + v2) = s1v1 + s1v2
(s1 + s2)v1 = s1v1 + s2v1
(s1s2)v1 = s1(s2v1)

So I claim the F field part to be significant because that's where the scalars come from. As example we could state that C (the set of complex numbers) is a two-dimensional vector space over a field F where F is R (the reals) with standard unit basis vectors e1 = 1 = (1, 0) and e2 = i = (0, 1). This example (from the book) helps me understand that the vector space and it's field can viewed as separate entities.

A field is defined earlier as being:

A field F is a commutative ring where every non-zero element has a multiplicative inverse. A field is closed under division by non-zero elements.

Matt

PS: TheMCB note I assume element O is the additive identity. Such that for any v element of V, we have v + O = O + v = v.

[TheMotorcycleBoy]

Highly impractical I know. I'm currently struggling with Group Theory in a third year Pure Maths course, having not touched the subject for 25 years, knowing that my younger self would waltz through it

Glad to have you onboard SH. I'm currently on homomorphisms, isomorphisms, group monomorphisms, automorphisms and epimorphisms. A real word salad indeed. The book has only just started to cough up any examples!

later Matt

[TheMotorcycleBoy]

Consumed a bit of a lunch break listening to this

https://www.youtube.com/watch?v=XPF5fe1WdKY

re. Group Homomorphisms. I love the way the youtube can supply complete on demand coverage of all the lectures we'd see 30-50 years ago on BBC2's Open University programs.

Matt