⬅ Sorting: Selection, bubble, and insertion sort

Announcements

• Welcome back at long last
  • If only it were on more pleasant terms :\
• No quiz today since, well, last time was the exam :B
• Notes about project 3’s methods
  • isFullSolution and reject are what you should work on first
    • those can be tested without doing anything else
    • they’re subtly different though - be sure not to reject solutions with empty cells!
  • extend should find the next available empty cell
    • if all cells are full, it should return null
    • once it finds an empty cell, it should put a 1 there
      • don’t forget to make a copy!
      • make a method to copy a board.
  • next should find the most-recently-placed number and make it 1 bigger
    • if the most-recently-placed number was a 9, it should return null
    • if the most-recently-placed number was a 1, it should make it a 2; 2 -> 3; 3 -> 4; etc.
      • again, don’t forget to make a copy!
  • keeping track of which is the “most-recently-placed” number can be tricky
    • there are many ways to do it
    • you are allowed to change the code (parameters to functions, etc)
    • you’re not even required to store the board as an int[][]!

Ordering

• A total ordering is a mathematical concept
  • Basically, it boils down to: “you can compare any two things; those two things will either be equal or one will come after the other; and comparison is transitive”
• Numbers have a total ordering
  • You can pick any two numbers
  • They will either be equal (3 = 3) or will come in one specific order (3 < 4, not 4 < 3)
  • Comparison is transitive (3 < 4 and 4 < 5, so 3 < 5)
• Words have a total ordering, too
  • Depending on the language, of course
• You can’t sort a collection of things that don’t have a total ordering!
  • Or in Java parlance, you can’t sort a collection of things that don’t implement Comparable.

Sorting

• Unordered data is useful in a lot of cases, but many situations require ordered data
• Sorting is putting things in some kind of order
  • Ordered data is easier to search through (remember binary search?)
  • Ordered data is easier for people to look at
  • Many algorithms - not just searching - require ordered data to work properly
  • So lots of effort has gone into making good (efficient!) sorting algorithms.
• Automated sorting actually goes back to the 1800s (!)
  • Punchcard sorters existed then and were used well into the 1960s and 1970s
  • Early uses were for the census, taxes, and payroll
  • An entire industry of “unit record equipment” did many of the things we use computers for today
    • Think of them as “the great-grandparent of spreadsheets and databases”
• A sorted array is such that:
  • For all indices $i$ and $j$, if $i < j$, then $A[i] ≤ A[j]$.
  • Don’t be fooled by the “less-than-or-equals” sign - this works for more than just numbers
    • And that could be a greater-than-or-equals sign and still work just as well.
  • Ascending order means that we start with the “smallest” value and go up
  • Descending order means the opposite: start with the “biggest” value
• Duplicate items require some consideration…
  • For simple things like numbers, duplicates are no issue
  • But what about people’s names?
    • If they have a family name and a given name…
    • If you have 5 people with the same family name, what order should they go in?
      • They’re not really duplicates…
    • We’ll come back to this next time

Finding a minimum

• Maybe you’ve done this before (401? 0007?)
• This is sort of like a “running total” algorithm
  • Keep a “smallest” value
  • Loop over array and compare each against the smallest
  • If the value in the array is smaller than the current smallest, it becomes the new “champion”
• How many steps does this take?
  • $O(n)$ iterations
• How many comparisons do we do?
  • $O(n)$ comparisons
• We can find any extremum (minimum, maximum, whatever) this way, and it’s always gonna be linear

Selection Sort

• We now know enough to make a simple sorting algorithm: selection sort.
  • It’s called this because we select one item from the array, over and over.
• Let’s say I have this array: $\{ 4, 1, 3, 4, 2, 6, 5 \}$
  • If I want to sort it in ascending order…
  • Which item do I want to select to be the first item in the array?
    • The smallest. The minimum.
    • So, 1.
  • Let’s make a second array that contains the sorted items: $\{ 1 \}$
    • And let’s remove 1 from the original array: $\{ 4, 3, 4, 2, 6, 5 \}$
  • Now we just repeat the process.
    1. Select the smallest (minimum).
    2. Remove it from the original array.

    3. Put it at the end of the new array.

       $$\begin{array}{|r|l|} \text{Unsorted} & \text{Sorted} \\ \hline \{ 4, 1, 3, 4, 2, 6, 5 \} & \{ \} \\ \{ 4, 3, 4, 2, 6, 5 \} & \{ 1 \} \\ \{ 4, 3, 4, 6, 5 \} & \{ 1, 2 \} \\ \{ 4, 3, 4, 6, 5 \} & \{ 1, 2 \} \\ \{ 4, 4, 6, 5 \} & \{ 1, 2, 3 \} \\ \{ 4, 6, 5 \} & \{ 1, 2, 3, 4 \} \\ \{ 6, 5 \} & \{ 1, 2, 3, 4, 4 \} \\ \{ 6 \} & \{ 1, 2, 3, 4, 4, 5 \} \\ \{ \} & \{ 1, 2, 3, 4, 4, 5, 6 \} \\ \end{array}$$
  • How much space do we need for each step?
    • Well the unsorted array is getting 1 smaller, and the sorted array is getting 1 bigger…
    • The space is constant.
    • So it seems silly to need 2 arrays.
• Let’s make it work in-place - no second array needed.
  • For i = 0; i < n; i++
    1. Select the smallest (minimum), starting at item i.

    2. Swap it with item i.

       $$\begin{array}{|c|l|l|} \text{i} & \text{Array} & \text{Notes} \\ \hline \color{green}{0} & \{ \color{green}{4}, \color{red}{1}, 3, 4, 2, 6, 5 \} & \text{green=i, red=minimum}\\ \color{green}{1} & \{ 1, \color{green}{4}, 3, 4, \color{red}{2}, 6, 5 \} & \\ \color{green}{2} & \{ 1, 2, \color{green}{3}, 4, 4, 6, 5 \} & \text{no swap needed.}\\ \color{green}{3} & \{ 1, 2, 3, \color{green}{4}, 4, 6, 5 \} & \text{"}\\ \color{green}{4} & \{ 1, 2, 3, 4, \color{green}{4}, 6, 5 \} & \text{"}\\ \color{green}{5} & \{ 1, 2, 3, 4, 4, \color{green}{6}, \color{red}{5} \} & \\ \color{green}{6} & \{ 1, 2, 3, 4, 4, 5, \color{green}{6} \} & \text{could actually stop at n - 2...}\\ \end{array}$$
  • This way lends itself to being written recursively very easily.
    • Find minimum of array[i to end]
    • Swap minimum with array[i]
    • Recurse with i + 1
• What is the runtime?
  • How many times does the outer loop run? $O(n)$
  • How long does it take to find the minimum? $O(n)$
  • How long does it take to swap two items? $O(1)$
  • If we multiply all those together, it’s $O(n^2)$
  • How many comparisons do we do?
    • $O(n^2)$ again, since we do $n$ comparisons to find the minimum on each of $n$ iterations

Bubble sort

• Here’s another simple algorithm
• We look for inversions: when a pair of items are in the wrong order.
  • So, if we saw 5, 3, that’s an inversion - cause 5 should come after 3.
• So here’s the algorithm:
  1. For i = 0; i < n - 1; i ++
     • If A[i] > A[i + 1]…
       • Swap them, and set i = 0. (start over!)
  2. When this loop exits, no inversions have been found, and the array is sorted.

• Seems kind of silly, but it does work:

  $$\begin{array}{|c|} \text{Red are inversions} \\ \hline \{ \color{red}{4, 1}, 3, 4, 2, 6, 5 \} \\ \{ 1, \color{red}{4, 3}, 4, 2, 6, 5 \} \\ \{ 1, 3, 4, \color{red}{4, 2}, 6, 5 \} \\ \{ 1, 3, \color{red}{4, 2}, 4, 6, 5 \} \\ \{ 1, \color{red}{3, 2}, 4, 4, 6, 5 \} \\ \{ 1, 2, 3, 4, 4, \color{red}{6, 5} \} \\ \{ 1, 2, 3, 4, 4, 5, 6 \} \\ \end{array}$$
• Notice how sometimes numbers will move backwards through the array
  • This is why it’s called bubble sort - the values sort of “bubble” through the array slowly
  • It doesn’t go strictly beginning-to-end like selection sort.
  • And the values don’t end up in their final positions when we move them.
• What is the runtime?
  • This is a little trickier, since the loop kind of starts over again and again
  • But if we think about it in the worst possible case…
    • We would find an inversion at i == 0
    • Then at i == 1
    • Then at i == 2…
    • And each time, we would have to look at i + 1 items
    • Until i = n
  • It’s that sort of “triangular” behavior we saw before: $O(n^2)$ again!
  • You can also think of it like: each number in the array might need to move $O(n)$ positions down, and there are $n$ numbers
• How many comparisons?
  • Again, it’s gonna be $O(n^2)$
  • Cause each of those $n$ numbers will have to move $O(n)$ positions, requiring the same number of comparisons

Insertion sort

• Let’s (try to) do better.
• Insertion sort is a bit like a combination between selection and bubble sort.
  • Like selection sort, we will have a “sorted part” of the array.
  • Like bubble sort, we will “slide” items around.
• Here’s the algorithm:
  1. Pick the first item from the “unsorted” part of the array.
  2. Insert that item into the “sorted” part in the correct order.

     • We “slide” it into place right-to-left, kinda like bubble sort.

       $$\begin{array}{|c|} \text{Red is sorted} \\ \hline \{ \color{red}{4}, 1, 3, 4, 2, 6, 5 \} \\ \{ \color{red}{1, 4}, 3, 4, 2, 6, 5 \} \\ \{ \color{red}{1, 3, 4}, 4, 2, 6, 5 \} \\ \{ \color{red}{1, 3, 4, 4,} 2, 6, 5 \} \\ \{ \color{red}{1, 2, 3, 4, 4,} 6, 5 \} \\ \{ \color{red}{1, 2, 3, 4, 4, 6,} 5 \} \\ \{ \color{red}{1, 2, 3, 4, 4, 5, 6}\} \\ \end{array}$$
• Think of it like picking books off a shelf, and sliding them into place in the right order
• What is the runtime?
  • It takes $O(n)$ time to “slide” each number into place
  • Therefore it’s still $O(n^2)$…
    • Aaargh!!!
• How many comparisons?
  • Again, $O(n)$ for each item, since it might have to be compared to every item to its left
  • So $O(n^2)$ overall…
• Just as a brain teaser: how could we find where to put the item in the sorted part faster?
  • We could use binary search
  • That lets us find the right location in $O(log n)$ time…
  • But it still takes $O(n)$ time to move the items after it over. So we haven’t improved things significantly.

This is frustrating

• It feels like we should be able to do better, somehow.
  • Well, we can.
  • Next time we’ll look at three new sorting algorithms
  • Two of them will be faster in general..
  • And one will be faster in a totally different way!