Linear time ordering

How fast can we sort?

It depends on model of what you can do with the elements.

e.g.

quicksort

worst case : T(n)=Θ(n2)T(n)=\Theta(n^2) randomized quicksort : T(n)=Θ(nlog2n)T(n)=\Theta(nlog_2n)

heapsort : always T(n)=Θ(nlog2n)T(n)=\Theta(nlog_2n)

insertion sort：T(n)=Θ(n2)T(n)=\Theta(n^2)

merge sort T(n)=Θ(nlog2n)T(n)=\Theta(nlog_2n)

Can we do better than Θ(nlog2n)\Theta(nlog_2n) ?

Comparison sorting(model)

only use comparisons to determine relative order of elements.

State the theorem , prove it.And then we’ll give a counter example in the sense

Decision-tree model

example: sort <a1,a2,a3a_1,a_2,a_3>



In general :<a1,a2,a3...ana_1,a_2,a_3...a_n>

each internal (non-leaf）node has a label of the form i:j

i,j∈\in {1,2.....n1,2.....n}

Decision trees model to model comparison sorts

one tree for each n

view algorithm as splitting whenever if makes a comparison.

tree lists comparisons along all possible instruction traces.

all possible results=leaf node=n!

running time:(#comparison)=length of path

worst-case run_time=height of the tree=n*(n-1)/2

lower bound on decision-tree sorting:Any decision tree that sorts n elements has height Ω(nlog2n)\Omega(nlog_2n)

Proof：

#leaves must be ≥n!\ge n!

height h ⇒\Rightarrow#leaves ≤2n\le 2^n

⇒n!≤2n\Rightarrow n! \le 2^n

⇒h≥log2n!\Rightarrow h\ge log_2{n!}

Stirling’s approximation:

n!≈2πn−−−√(ne)n n! \approx \sqrt{2\pi n}{(\frac{n}{e})}^n



Merge sort and heapsort are asymptotically optimal.

Sorting in linear time

Counting sort:

Input:A[1….n] 1< A[i] < k 1≤i≤n\le i \le n

Output:B[1….n]=sorting of A

Auxiliary storage:C[1…k]

program

Example

Running Time: T(n)=O(k+n)T(n)=O(k+n)

if k=O(n)k = O(n) T(n)=O(n)T(n)= O(n)

Stable sort preserves the relative order of equal elements.

Radix sort

Radix sort[Herman Hollerith about 1890]

Example

Correctness

induct on digital position t

assume by induction : sorted on lower-order t-1 digits

sort on digital t

⇒\Rightarrow

- if two elements have the same tthdigitt^{th} digit

Stability ⇒\Rightarrow same order

Induction hypothesis ⇒\Rightarrow sorted order

- different tthdigitt^{th} digit ⇒\Rightarrowsorted order

Analysis

use counting sort for each digit.T(n)=O(k+n)

say n integers,each b bits. (Range：0~2b−12^b-1)

sort 1 bit in a time.

split into b/r “digits” ,each “digits” r bits.

b/r is the number of rounds.

The maximum of every “digits” is 2r−12^r-1

Running time:T(n)= b/r*Every round time

Every round time uses counting sort : n numbers,range 0:(2r−12^r-1)

Every round time = O(n+2r2^r)

Running time:T(n)=O(b/r*(n+2r2^r))

Take the derivative of b/r*(n+2r2^r) by r.

⇒O(b∗n/log2n)\Rightarrow O(b*n/log_2n)

if numbers in range 0:2b−10:2^b-1(0:ndn^d-1)

⇒O(d∗n)\Rightarrow O(d*n)

If you have arbitrary integers that are one word length long.

The best algorithm we know for sorting runs in O(nlog2(log2n)−−−−−−−−−√)O(n\sqrt{log_2({log_2n})} )

worst case:nlog2(log2n)−−−−−−−−−√n\sqrt{log_2({log_2n})}

Code for CountingSort:

code in Github