The Story of Dimension
Have you heard a geek friend seem to speak Greek? “I want to change a tensor from the shape of (2, 3, 4, 5) to the shape of (4, 5, 3, 2).” What does that mean?
You’ll find it’s very simple.
Part 1a - An innovation
With many items in her bag, it was hard for Monkey to search for something, say her sunglasses. One day, she thought of an “innovation:” put two small bags inside her big bag to better organize her stuff! She placed her personal items in one small bag and food in the other small bag. When searching for her sunglasses, she looked inside her bag, “Hmm, two small bags. Neat! Let me take a look in the personal items bag.” She’s happy about her innovation!
“You added a dimension” is how computer geeks would talk about Monkey’s innovation. Dimension? It just means size, i.e. “You added a different size of bag.” [Can it be a different size in addition to the existing bag, or does it have to nest inside the existing bag?]
Geeks use a pair of brackets to indicate a bag. [‘sunglasses’, ‘wallet’] means putting sunglasses and a wallet inside a bag. Of course, [‘apple’, ‘orange’] means putting an apple and an orange inside a bag. How do they show that they are putting these two bags inside a bigger bag? Simple, add another pair of brackets:
[ [‘sunglasses’, ‘wallet’], [‘apple’, ‘orange’] ]
[“sunglasses”, “wallet”] is called a tensor, or array.
[ [‘sunglasses’, ‘wallet’], [‘apple’, ‘orange’] ] is also a tensor.
This is the idea that you can “bag stuff,” whether you place “sunglasses” and “wallet” in a bag or place smaller bags in a big bag.
So these are Monkey’s tensors: ]
[ ['grey phone', 'grey wallet', 'grey sunglasses', 'grey key'],
['cucumber', 'peas', 'sweet potato', 'apple'] ]
Part 1b - Why a tensor?
Monkey’s stove top has four burners. She loves cooking four similar foods at the same time, such as four stir-fry dishes. By using all four burners, the cooking time is reduced from 40 minutes to 10 minutes. Different stir-fries use different ingredients and different seasonings, but the operation is similar: move food around in a wok with a paddle.
Yes, it’ll save time if you group similar tasks and perform, or process, them together. Yes, a tensor indicates similar items that can be processed together, or “parallel processed.”
Stir-fry these dishes at the same time, in parallel: [‘celery’, ‘asparagus’, ‘green bean’, ‘chicken breast’]
Monkey went to a school with a strict [test? performance?] scoring system. For this quarter’s three classes, she got 3, 3, and 2. She usually told people, “Although my scores are 3, 3, and 2, they’re equivalent to 4, 4, and 2.7 in other schools”.
She multiplied each of her scores by 1.3. A special type of hardware [a chip?] can calculate, or process, numbers in parallel, similar to a stove with multiple burners.
[3, 3, 2] * 1.3 shows that you are multiplying each number in the bracket by 1.3 in parallel. Who doesn’t want to talk about his or her achievements accurately?
Parallel processing is important. Imagine you’re cooking for a feast, say 12 similar dishes. It takes 10 minutes to parallel stir-fry four dishes. It takes 30 minutes to cook 12 dishes in three batches. If you work in a commercial kitchen with 12 burners, it takes only 10 minutes. for all 12 dishes. Had you cooked one dish at a time, it would have taken two hours!
You said processing means performing a task, and parallel processing means performing similar tasks together--but what does “processing” mean in the context of AI?
--> number of burners is the size of the batch
big batch = 12 burners
Part 1c - Many dimensions
When Monkey added two small bags to help organize her bag, she had two sizes of bags, even though the total number of bags was three. Two sizes of bags? Two dimensions. Yes, dimension means the number of different sizes of bags, not the number of bags.
If someone has many bags, is it impressive? Not really. If someone has many different sizes of bags, is it impressive? Not really, either.
Of course, one could have bags of many different sizes, hmm, I mean many dimensions, from the biggest bag to the smallest bag.
After Monkey’s proud boast, Jade and Nightingale, also began to park their stuff using small bags.
They got one big bag to hold a bag for each of the three housemates, before renting a minivan for a weekend getaway with friends from another apartment in the city. It’s quite handy to load or unload just one big bag at each apartment.
Their big bag contains this stuff:
[ [ ['grey phone', 'grey wallet', 'grey sunglasses', 'grey key'], ['cucumber', 'peas', 'sweet potato', 'apple'] ],
[ ['green phone', 'green wallet', 'green sunglasses', 'green key'],
['radish', 'Adzuki beans', 'rice', 'orange'] ],
[ ['brown phone', 'brown wallet', 'brown sunglasses', 'brown key'],
['celery', 'chickpeas', 'oatmeal', 'avocado'] ] ]
Let’s name these three sizes of bags:
bag for an apartment
bag for one person
bag for one category (whether it’s personal items or food).
Friends from another city want to go as well! So they get one huge bag (bag for a city) to hold the stuff belonging to one city and rent a minivan.
This can continue. The bag for a state, the bag for a country, the bag for the earth, the bag for the solar system, etc.
So far we have mentioned eight sizes of bags, i.e. eight dimensions:
bag for the solar system
bag for a planet
bag for a country
bag for a state
bag for a city
bag for an apartment
bag for a person
bag for a category
The bag for the solar system is handy when people from Earth have a vacation far far away!
If someone has many bags, is it impressive? Not really. If someone has many sizes of bags, is it impressive? Not really, either.
So if geeks tell you they’re working on “100 dimensions”, it’s nothing complicated. It just means they’re into organizing things.
A tensor can have any number of dimensions.
By the way, a one-dimension tensor is also called a vector. A two-dimension (2D) tensor is also called a matrix. We mention these names so you know what people are talking about. It’s much simpler to always refer to them as a tensor. If you want to give more details, say it’s a 2D tensor.
So these are Monkey’s tensors: [ ['grey phone', 'grey wallet', 'grey sunglasses', 'grey key'], ['cucumber', 'peas', 'sweet potato', 'apple'] ] So are there three tensors??
Part 1d - same number of items for same size bag
Most of the students in Monkey’s school took three classes each semester. Some took two classes, while a few took four classes in one semester. Since the scores given by the school are very low, they must be multiplied by 1.3 to be comparable to other schools’ scores.
For a student who took two classes:
[2, 2] * 1.3
For a student who took three classes: [3, 3, 2] * 1.3
It’s faster for the hardware to process tensors with the same number of scores in each [tensor?]. Since the highest number of scores anyone gets in a semester is four, we could pad the scores of all the other students, or add scores of zero, to give them four scores too. We can add zeros to the tensors if the number of scores received is less than four, like this:
[2, 2, 0, 0]
[3, 3, 2, 0]
[3, 3, 2, 3]
We can then multiply each tensor by 1.3, like this:
[2, 2, 0, 0] * 1.3
[3, 3, 2, 0] * 1.3
[3, 3, 2, 3] * 1.3
Now the number of scores to be multiplied for each student is four. In other words, the same size bags contain the same number of elements. What’s an element in AI? So the size of a tensor the same thing as the number of elements in the tensor, right?
Part 2 - shape
How do you describe a tensor, i.e. bagged stuff? In addition to knowing the number of sizes of bags, you want to know the number of items in each size bag.
For example, Monkey’s bag contains two small bags. Each small bag contains four items. (We always make the same size bag contain the same number of items, using padding.)
Remember, the number of sizes of bags simply means the dimension. The number of contents is a content an element? in each of the two dimensions is two and four. (2, 4) is called the shape of a tensor. Remember, Monkey’s bag represents the tensor. In other words, the shape means =shows? the number of contents assigned to each different size bag., expressed as xxx.
The shape is a description of a thing, not the thing, right? You said “How do you describe a tensor?” and “the shape is the number of contents” But a shape not the number of contents, it’s a description of the numbers of contents within a tensor, right? That is, a shape is a set of numbers in parentheses that describes a tensor?
Let’s look at another example. Monkey’s apartment bag contains 3 medium bags for Monkey, Jade, and Nightingale respectively. Each medium bag contains 2 small bags. Each small bag contains 4 items. Let’s write down the numbers: (3, 2, 4). It’s the shape of a tensor, i.e. Monkey’s apartment bag.
Dimension 0 refers to the big size and dimension 1 refers to the small size.
You said dimension means number of sizes of bags. If a dimension is a number, what does “big size” mean?? Oh you do explain this in Part 3 -- I suggest omitting this sentence.
Geeks place the same number of things in a bag. which bags? So it’s easy to compute total number of items. A big bag contains 2 small bags. Each small bag contains 4 items. So total number of items are 8.
Part 3 - start with size 0 bag
When something is challenging, Monkey will say “it’s fun”. As long as someone uses words consistently, it’s easy to know what they mean.
Geeks always call the biggest bag dimension 0. If there are 2 sizes, they will call it dimension 0 and dimension 1, with dimension 0 refers to the biggest size of bag. If there are 3 sizes, they will call it dimension 0, dimension 1, and dimension 2, with dimension 0 refers to the biggest size of bag.
They have many different names. What has many different names?
This is a two dimension (short name is 2D):
[['grey phone', 'grey wallet', 'grey sunglasses', 'grey key'],
['cucumber', 'peas', 'sweet potato', 'apple']]
Dimension means number of sizes of bags, so 2D means two sizes of bags, right?
The shape is (2, 4)
Dimension 0 is also called axis 0, or axis x.
Dimension 1 is also called axis 1, or axis y.
Dimension 2 is also called axis 2, or axis z.
“Dimension” and “axis” have plain meanings in English, so it might be good to have more explanation here. Are the terms “dimension” and “axis” interchangeable?
For dimension 0 (the biggest bag, within the outermost bracket), it’s also called row
What is also called row? is a dimesion a row or is it just the biggest dimension a row?
2 rows, 4 columns
For dimension 0, it’s also called height
For dimension 1, it’s also called width
height is 2, width is 4.
This is a three-dimension (shore name is 3D): This is a three-dimension what? What’s a shore?
[[['grey phone', 'grey wallet', 'grey sunglasses', 'grey key'],
['cucumber', 'peas', 'sweet potato', 'apple']],
[['green phone', 'green wallet', 'green sunglasses', 'green key'],
['radish', 'Adzuki beans', 'rice', 'orange']],
[['brown phone', 'brown wallet', 'brown sunglasses', 'brown key'],
['celery', 'chickpeas', 'oatmeal', 'avocado']]]
For dimension 0, it’s also called depth
For dimension 1, it’s also called height
For dimension 2, it’s also called width
This is the opposite of what you said before: dimension 0 is height and dimention 1 is height, dimension 1 is width and dimension 1 is height - totally confused
The shape is (3, 2, 4)
Part 4 - Why switch axis? axes is the plural
Monkey thought of putting all foods in a bag. When arriving at a place, she can put the entire bag in the refrigerator.
So she re-organize the bags:
[[['grey phone', 'grey wallet', 'grey sunglasses', 'grey key'],
['green phone', 'green wallet', 'green sunglasses', 'green key'],
['brown phone', 'brown wallet', 'brown sunglasses', 'brown key']],
[['cucumber', 'peas', 'sweet potato', 'apple'],
['radish', 'Adzuki beans', 'rice', 'orange'],
['celery', 'chickpeas', 'oatmeal', 'avocado']]]
There are other reasons to reorganize the elements? to place the elements along different axes? too, such as people which people? data scientists or data users? are more familiar with certain way. of presenting the data? The method requires it. What method requires what? meaning the method you’re using to process the data determines the elements you should put in a tensor?
Part 4b - make a small program to do automatic conversion
Use HWSV, more clear
Def convert(old, new):
Return (1, 0, 2)
Num - school example
Compute mean
All Greek to me :)
Part 5a - different operations over an axis
what is an operation and is it the same as aa process?
The school has two classes, class 1 and class 2, and each class is taught by a different teacher.
Student 1A, music, English
Student 1B
Student 1C
Student 2A
Student 3B
Student 2C
#x = np.array([[[2, 4], [2, 4]], [[2,3], [3, 4]], [[2, 3], [2, 4]]])
Three students, each student took 2 classes
>>> x = np.array([[3, 4], [3, 3], [3, 2]])
>>> x.mean(axis = 0)
array([3., 3.])
>>> x.mean(axis = 1)
array([3.5, 3. , 2.5])
>>> x.mean(axis = 0) the biggest axis, per class
array([3 , 3.5, 4 ])
>>> x.mean(axis = 1)
array([4., 3.])
The code needs explanation. From here on, where you show the actual code, you could give your data scientist readers some sort of reference, so they know what code you’re using or understand the format?
Part 5b - Switch dimensions, axis
The rightmost number in the shape is the smallest bag
Person, category, (importance) who is the person, how do you determine category, importance of what -- and why is importance in parentheses
(3, 2, 4)
Person
Category
Importance
Axis (0, 1, 2)
category, person, (importance)
Category
Person
Importance
(2, 3, 4)
Axis (1, 0, 2) wait, I thought an axis was a dimension-- how can it be three numbers
Start with a normal one a normal what,
(0, 1, 2, 3)
Smallest dimension (at pos 3) has 2 items is pos a technical term? if not, maybe use whole word
dimension at pos 2 has 126 126 whats? items, contents, elements?
dimension at pos 1 has 2096
dimension at pos 0 has 1
[1, 2096, 126, 2]
[2096, 126, 1, 2]
(1, 2, 0, 3)
The smallest dimension, no change where is 1,2,0,3?? why
They will have comments, hwkv -> vkhw
(2, 2, 10, 3)
Look at the actual items.
Summary
Dimension = size no!
Say “this tensor has 3 dimensions” instead of using other terms
Say “size 0 is the biggest dimension or axis”, computation over the axis, split, stack is easier to understand
Switching axis axes is just a different way to organize stuff. You use the Same what? search, computation, sorting? kinds of methods that are already written coded?. You need to convert/transform convert what? and what does convert or transform mean, switching axes?