Category Archives: Computer Science

Prim’s Algorithm

ref –

  • https://www.geeksforgeeks.org/travelling-salesman-problem-set-2-approximate-using-mst/
  • https://iq.opengenus.org/graph-representation-adjacency-matrix-adjacency-list/
  • https://www.gatevidyalay.com/prims-algorithm-prim-algorithm-example/
  • source – https://github.com/redmacdev1988/PrimsAlgorithm
  • https://www.tutorialspoint.com/Prim-s-algorithm-in-Javascript

What it solves:

Traveling Salesman Problem (Approximate using Minimum Spanning Tree)

Algorithm:
1) Let 1 be the starting and ending point for salesman.
2) Construct MST from with 1 as root using Prim’s Algorithm.
3) List vertices visited in preorder walk of the constructed MST, and add 1 at the end.

Let us consider the following example. The first diagram is the given graph. The second diagram shows MST constructed with 1 as root. The preorder traversal of MST is 1-2-4-3. Adding 1 at the end gives 1-2-4-3-1 which is the output of this algorithm.

Given a problem like so, we use Prim’s algorithm to create a minimum spanning tree for a weighted undirected graph. It finds a subset of the edges that forms a tree, where the total weight of all the edges in the tree is minimized.

What is Minimum Spanning Tree?

Given a connected and undirected graph, a spanning tree of that graph is a subgraph that is a tree and connects all the vertices together. A single graph can have many different spanning trees. A minimum spanning tree (MST) or minimum weight spanning tree for a weighted, connected and undirected graph is a spanning tree with weight less than or equal to the weight of every other spanning tree. The weight of a spanning tree is the sum of weights given to each edge of the spanning tree.

Prim’s Algorithm

Given Problem

Adjacency List

We create an adjacency list. From the diagram, we see that A connects to B and C. So we create key A, and then have it reference a Linked List with nodes B and C.

We see B connects to C, E, and D. So in our adjacency list, we add key B, and have it reference a Linked List with nodes C, E, and D. Repeat until all edges and vertices have been added.

A -> [B] -> [C]
B -> [A] -> [C] -> [E] -> [D]
C -> [A] -> [B] -> [E]
D -> [B] -> [E] -> [F]
E -> [D] -> [E]

The key to picking only the minimum path and getting rid of the rest is accomplished by:

  • minimum priority queue
  • visited dictionary to keep track of all visited nodes

The Minimum Queue

We use the minimum priority queue to keep all the edges from nodes that we have visited. We group these edges into this queue, then take the minimum of it. That is how this algorithm picks the minimum path to travel.

Start

We start at node A so we draw it.

We need to process this node A in our min priority queue so we should add it to initialize our queue.


Min Queue: [A]
Visited: {}

Now that the queue has an item, we can start off the process it.

We pop A from the queue to process it.


Min Queue: []
Visited: {}

Has it been visited? No, so we put it in Visited dictionary.
We see from the adjacency list that A connects to B (with distance of 2) and C (with distance of 3).
So we push B and C into our queue.


Min Queue: [ B(A, 2), C(A, 3) ]
Visited: { “A” }

This means that all edges from A is only B and C. And from this group of edges, we use the min priority queue to get the absolute minimum path (which is 2) at node B.

Hence pop B from the queue.


Min Queue: [ C(A, 3) ]
Visited: { “A” }

Has B been visited? We look at the Visited dictionary. No. So we then add B to the visited dictionary. . We draw our result:


So we get all its edges from the adjacency list. We see that its nodes D(B, 3), C(B, 5), and E(B, 4).


Min Queue: [ C(A, 3), D(B, 3), C(B, 5), E(B, 4) ]
Visited: { “A”, “B” }

Now we see that the number of available paths have increased to four. We then get the minimum path, which is C(A,3). Has C been visited? Nope, we add it to the Visited dictionary. We draw the path in our diagram.

Then we process it by getting all its edges from the adjacency list. We see that it has node of E(C, 4). We push it onto the Min Queue.
Now, we have 4 available paths from nodes A, B, and C.


Min Queue: [ D(B, 3), C(B, 5), E(B, 4), E(C, 4) ]
Visited: { “A”, “B”, “C” }

We get minimum of those paths which is D(B,3). Has D been visited? Nope, we add it to the visited dictionary. We draw the path in our diagram.

Then we process D by getting all its edges from the adjacency list. We see that it has node of E(D, 2), F(D, 3). We push it onto the Min Queue.


Min Queue: [ C(B, 5), E(B, 4), E(C, 4), E(D, 2), F(D,3) ]
Visited: { “A”, “B”, “C”, “D” }

We pop the minimum of the priority queue: E(D, 2). Has E been visited? Nope, we add it to the visited dictionary. We draw the path in our diagram.

Then we process E by getting all its edges from the adjacency list. We see that it has node of E(F, 5). We push it onto the Min Queue.

Hence at this point, we have 5 available edges from our visited nodes.


Min Queue: [ C(B, 5), E(B, 4), E(C, 4), F(D,3), E(F, 5) ]
Visited: { “A”, “B”, “C”, “D”, “E” }

We pop the minimum which is F(D, 3), and process it. Has F been visited? Nope, we add it to the visited dictionary. We update our answer diagram:

From our adjacency list, we see that F does not have a Linked list of edges. Hence, we DO NOT PUSH ANYTHING onto our Priority Queue:


Min Queue: [ C(B, 5), E(B, 4), E(C, 4), E(F, 5) ]
Visited: { “A”, “B”, “C”, “D”, “E”, “F” }

We keep processing. We pop from the minimum queue: E(B, 4) . Has it been visited? Yes, so we skip and continue dequeueing.


Min Queue: [ C(B, 5), E(C, 4), E(F, 5) ]
Visited: { “A”, “B”, “C”, “D”, “E”, “F” }

We pop from the minimum queue: E(C, 4) . Has it been visited? Yes, so we skip and continue dequeueing.


Min Queue: [ C(B, 5), E(F, 5) ]
Visited: { “A”, “B”, “C”, “D”, “E”, “F” }

We pop from the minimum queue: C(B, 5) . Has it been visited? Yes, so we skip and continue dequeueing.


Min Queue: [ E(F, 5) ]
Visited: { “A”, “B”, “C”, “D”, “E”, “F” }

We pop from the minimum queue: E(F, 5) . Has it been visited? Yes, so we skip it and continue.

Now our queue is empty and we’re done.

The answer is:

Graphs Problems (networks, routes/paths, scheduling…etc)

ref –

  • https://www.geeksforgeeks.org/depth-first-search-or-dfs-for-a-graph/
  • https://www.geeksforgeeks.org/breadth-first-search-or-bfs-for-a-graph/
  • https://stackoverflow.com/questions/14720691/whats-the-purpose-of-bfs-and-dfs

BFS and DFS are graph search algorithms that can be used for a variety of different purposes. Many problems in computer science can be thought of in terms of graphs. For example, analyzing networks, mapping routes, and scheduling are graph problems. Graph search algorithms like breadth-first search are useful for analyzing and solving graph problems.

One common application of the two search techniques is to identify all nodes that are reachable from a given starting node. For example, suppose that you have a collection of computers that each are networked to a handful of other computers. By running a BFS or DFS from a given node, you will discover all other computers in the network that the original computer is capable of directly or indirectly talking to. These are the computers that come back marked.

BFS specifically can be used to find the shortest path between two nodes in an unweighted graph. Suppose, for example, that you want to send a packet from one computer in a network to another, and that the computers aren’t directly connected to one another. Along what route should you send the packet to get it to arrive at the destination as quickly as possible? If you run a BFS and at each iteration have each node store a pointer to its “parent” node, you will end up finding route from the start node to each other node in the graph that minimizes the number of links that have to be traversed to reach the destination computer.

DFS is often used as a subroutine in more complex algorithms. For example, Tarjan’s algorithm for computing strongly-connected components is based on depth-first search. Many optimizing compiler techniques run a DFS over an appropriately-constructed graph in order to determine in which order to apply a specific series of operations. Depth-first search can also be used in maze generation: by taking a grid of nodes and linking each node to its neighbors, you can construct a graph representing a grid. Running a random depth-first search over this graph then produces a maze that has exactly one solution.

This is by no means an exhaustive list. These algorithms have all sorts of applications, and as you start to explore more advanced algorithms you will often find yourself relying on DFS and BFS as building blocks. It’s similar to sorting – sorting by itself isn’t all that interesting, but being able to sort a list of values is enormously useful as a subroutine in more complex algorithms.

We use graphs to present connected nodes.

Graph consist of nodes and edges.

Adjacent – if two nodes are directed connected. They are neighbors.

If it’s a directed graph, and John has an arrow towards Bob, then John is adjacent to Bob. But opposite not true.

Edges also have a weight. If two people interact a lot, they can have a higher weighted edge.

Finding two shortest path to two nodes.

Adjacency Matrix

An adjacency list is a collection of unordered lists used to represent a finite graph.

Each list describes the set of neighbors of a vertex in the graph.

Let’s do Depth First Search on our example graph

The idea is to go forward in depth while there is any such possibility. If not then backtrack.

Whenever we push an unvisited node, we print it.

Before we start, there are two assumptions:

– we start at node C
– nodes are inserted in alphabetical order.

We use a stack.

So we start at C. We push C onto our stack, and print it.

stack: C
output: C

C is connected to three nodes, A, B, and D. We do this alphabetically so we push A onto the stack, and print it.

stack: C A
output: C A

At A, we have two nodes, B and E. We push B onto our stack, and print it.

stack: C A B
output: C A B

B has one neighbor E. So we push E onto the stack, and print it.

stack: C A B E
output: C A B E

Since E has no connecting neighbors, we pop it.

stack: C A B
output: C A B E

We are now at B. B’s only neighbor of E has already been visited so we pop B.

stack: C A
output: C A B E

A’s neighbors B and E have all been visited. So we push A.

stack: C
output: C A B E

Now we’re at C.

C’s A has been visited, and B has also been visited. But B is unvisited so we push D onto the stack, then print it.

stack: C D
output: C A B E D

D has no unvisited nodes so we pop it.

stack: C
output: C A B E D

C has no unvisited nodes so we pop.

stack:
output: C A B E D

Stack is empty so we’re done.

Source Code

We create a list that has a tail pointing to the latest item we insert. That way, everything is order. Whatever you added first, will be processed first as head is referencing it. Tail will always reference the very last item you add.

Create a Graph with x number of nodes.

We use recursion on the local stack frame in order to simulate a stack.

Each list that is referenced by an Adj[key] is pushed onto the stack. As we process each node down this list, we get more keys, say key2.

We then push list referenced by Adj[key2] onto the local stack frame by recursively calling the function. We keep doing this until we exhaust this list.

Then the function gets popped off the local stack frame.

We look at our graph, and add the neighbors that are connected to our node. We always do this before we run the DFS function.

After adding all the nodes, and specifying how many nodes we have in the constructor, we are ready to call this function.

print our adjacency list and the lists that they contain

print the output array

Breadth First Search on the graph

The trick is that you visit a dequeued node. If that node is NOT visited, we add its neighbors to the queue. If that node HAS ALREADY BEEN visited, we keep dequeueing.

We start at C, so we add C to the queue.

queue: [C]

We remove C from the queue and visit it.

output: C
queue: []

We now must add C’s unvisited neighbors tot he queue (in alphabetical order).

output: C
queue: [A, B, D]

We visit A, and remove it from the queue.

output: C A
queue: [B, D]

Since we visited A, we must add A’s unvisited neighbors to the queue. Those neighbors are B and E.

output: C A
queue: [B, D, B, E]

We visit B, and dequeue it.

output: C A B
queue: [D, B, E]

Since we visited B, let’s look at its neighbors. It has one neighbor E. Let’s add it to the queue.

output: C A B
queue: [D, B, E, E]

No we visit D and print. Then dequeue it.

output: C A B D
queue: [B, E, E].

since we visited D, we must add its unvisited neighbors E.

output: C A B D
queue: [B, E, E, E]

Now we dequeue B, but we see that we have already visited it.

output: C A B D
queue: [E, E, E]

We then dequeue E. It has not been visited so we output it.

output: C A B D E
queue: [E, E]

E has no neighbors so we don’t add anything to the queue.

We then dequeue again. Has no neighbors so we continue to dequeue.

output: C A B D E
queue: [E]

Finally, we dequeue the last E. It has already been printed. And it has no neighbors. The queue is now empty so we’re done.

output: C A B D E
queue: []

We first create a queue.

We use the Linked List from the previous example.

So with Breadth first traversal of the graph, the trick is that we start at the initial node.

We then take all the neighbors of that node and add them to the queue. Once we added all those neighbors, we pop the initial node, and then move on to process the next one in queue.

So as you can see, we give it an initial key to start from. In our case it is ‘c’. So fromNodeKey is ‘c’.
We then keep track if they are visited or not via a visited array.

Initially, we only have one node in the queue. So we start processing the while loop. We pop the ‘c’ node and then push all the nodes from its neighbors via an inner while loop.

In this inner loop, we check to see if the key has been processed by associative array access on the visited array. If it has not been visited, we mark it visited and then add it onto our queue to be processed later.

Source Code

Depth First Graph Traversal

Breadth First Traversal for Graphs

Linked List used is same as above.

Mosh Trees Tutorial

Non Linear Data structures – trees

Tree is a data structure that stores element in a hierarchy.

Root – Top node of tree
Root is parent of its children.
Each node can have up to two nodes…this is called a binary tree.

Trees

– represent hierarchical data
– databases
– autocompletion
– compilers (syntax tree to parse expressions)
– compression (JPEG, MP3)

left < node < right At every step of comparison, we throw away half of the tree. This is what we call Logarithmic Time

Lookup
O(log n)

Inserting node
Finding a Node O(log n) + Inserting Node O(1) = O(log n)

Delete Node
O(log n)

Depth First Search (DFS)

pre-order – (left of node)
in-order – (bottom of node) ascending order
post-order – (right of node)

Breadth First – Level by level (BFS)

Recursion

Iteration – loops.
Recursion – no loops.

Base condition terminates Recursion.

Depths and Height

Depths starts at the root

Depths of root is 0. As it goes deeper, it increases.
Depth of 10 is 1.
Depth of 21 is 2.
Depth of 8 is 3.

Depth – # of edges from root to target node.

Height is opposite of depth.

Height starts at the leaf

Height of leafs are 0.
As we go up, height increases. Longest path from target node to leaf.

Height of 6 is 1.

Height of 30 is 1.

Height of 10 is 2. We actually have 3 different paths. 3 to 10. 8 to 10. 21 to 10. Longest path is 2.

Height of 20 is 3, because it’s the longest path.

1 + max(height(L) + height(R))

Post Traversal – visit all leaves first

Find minimum value in BST

If it’s a standard BST, use recursion, and get leftmost leaf.

But if its NOT a BST, values are all over the place, so

Exercises

– height of tree. return Math.max of left subtree and right subtree. That way, you get the height at the root.

– # of nodes at distance k. simple pre-order and print.

– isBST? pass in parameter min, max. Make sure node value stays within range. If not, false. Kick off with -Infinity and +Infinity.

Minimum – isBST? if yes, get leftmost. If not, then must have parameter reference and recurse down, do compare with every value.

Maximum – isBST? if yes, get rightmost. If not, then must have parameter reference and recurse down, do compare with every value.

– Are the two tree equal (Equality Check)?

– traverse BST (Level Order Traversal)

for i to height of tree
let arr = getNodesAtDistance(i)
for j to arr.length
log arr[j];

Additional exercises

1- Implement a method to calculate the size of a binary tree. Solution: Tree.size()

2- Implement a method to count the number of leaves in a binary tree. Solution: Tree.countLeaves()

4- Implement a method to check for the existence of a value in a binary tree using recursion. Compare this method with the find() method. The find() method does the same job using iteration. Solution: Tree.contains()

5- Implement a method to check to see if two values are siblings in a binary tree. Solution: Tree.areSibling()

6- Implement a method to return the ancestors of a value in a List. Solution: Tree.getAncestors()

SOLUTION

Test

Hash Table (Linear Probing)

ref – https://github.com/redmacdev1988/LinearProbeHashTable
https://stackoverflow.com/questions/9124331/meaning-of-open-hashing-and-closed-hashing

Linear Probing is an Open Addressing scheme for hash tables.

The “open” in “open addressing” tells us the index (aka. address) at which an object will be stored in the hash table is not completely determined by its hash code. Instead, the index may vary depending on what’s already in the hash table.

The “closed” in “closed hashing” refers to the fact that we never leave the hash table; every object is stored directly at an index in the hash table’s internal array. Note that this is only possible by using some sort of open addressing strategy. This explains why “closed hashing” and “open addressing” are synonyms.

Best case – O(1).
For insertion, it hashes to an empty element. We insert. done.
For search, it hashes to its initial location. We do a check. Keys match. done.
Deletion – Searches for the element with the key. We find it. We remove it. The next element is empty so we’re done.

Worst case – all are O(n), because the worst possible case scenario is that we have to iterate the whole array after the initial collision.

Insertion

Insertion for linear probe deals with 2 steps:

1) hash to a location. If the location is empty, done.
2) If the location is occupied, we probe for an empty space.

Probe involves going step by step down the cells and trying to find an empty cell. Once found, we place the object there.
Note that we do not circle around back to the beginning of the array. If no empty cells are to be found, we return -1. Other alternatives can be to grow the array and try inserting again.

Retrieving the object

Searching for a cell with a certain key involves starting at the hashed index:

1) If the keys match, then return the cell. If no match, keep going forward (probe) and looking for a match on our key.
2) If an empty cell is found, the key is not in the table. If the key did exist, it would be placed in the empty cell first, and not have gone any further in the later cells. This is because during insertion, when a value is hashed and there is a collision, it would naturally probe until it finds an empty spot to put its object. Thus, when searching, hitting an empty spot means its the end of the line for a particular item that was hashed to a previous location.

Deleting an Object

Deleting an item is the same as searching. When we have successfully found the cell it delete, we:

1) remove the cell so the array element references null
2) Here’s the important part. Once the element was is removed in 1), we need to find a replacement down the line. The reason why we need to find a replacement is shown below.

Say we want to insert key “job”. It gets hashed to index 6, then we move it to the next empty space, which is index 9.

The next operation is that we want to remove key “last name”. So last name gets hashed to index 6. We go down the array until we find “last name” at index 8. We remove it.

Now, say we want to search for key “job”. It would get hashed to index 6. Then it would keep going and then it would hit index 8 on an empty cell. It would stop and say key “job” does not exist. However, clearly, job exists down the line at index 9.

Solution on what to do for the empty space after deletion

When deleting, we remove the cell, then:

1) then go down the list to find a replacement for this empty cell
2) The replacement must have a key equal or smaller than the recently deleted item.

This is because all items are inserted on or after its initial hashed index. Thus, we can replace the empty cell with hashed index that is smaller because the search is coming from the left.

But why? take a look at an example:

Sa we want to delete key “last”, which was initially hashed to index 6, and probed to index 7. Now, from the previous example, we know we can’t just let it go like this.

We must go down the line and find a replacement. The notion is that we must find a replacement that has key <= our key. But what if we dont' do this? What will happen? Say we simply get the next available replacement, which in our case, is key "name" (initially indexed to 8). So as you can see, say we bring key "name" over to index 7. Then, we make a search for key "name". Its initial hash index would be 8. And it wouldn't find anything there. If data does get placed there, it wouldn't be key "name" that we're looking for. Let's see what would happen if we iterated the array for a cell that has a key <= to our key. Picking an element with hash <= removed element's hash

So after removing key “last”, we can’t just get any element to replace it. We must find an element with a hash <= our hash. Since we’ve removed “last” with key of 6, we then iterate the array to find that element.

We go down the list, name has hash 8, nope. job has hash 8. nope. age has has 6. Perfect! so we move the element with key “age” and hash 6 to index 7. Don’t forget that we must do this continuously until we hit an empty slot or the end of the array.

Now, when you try to search for “age”, it will hash you to index 6, which you collide with key first. Then you probe. At index 7, we have found the element with key “age”. done!

Problems of Linear probing – Cluster

Due to linear probing, clusters may form. Its when we keep hashing to an index and then linearly adding items down the line. A cluster will start forming at the hash index and future hashes will be unwieldy because every future hash will have to step through that cluster.

To solve this problem, we use Quadratic Probing.

((hash(key) + i)^2) % TABLE_SIZE

How will it prevent from probing?

With linear probing, we increase step by step. But if we step quadratically, our steps will widen, and this spread the insertions out.

However, the problem here is that Quadratic Probing will circle back and it keeps making the same jumps. This ends up with infinite loop.

Double Hashing

Given,
prime is a prime # smaller than TABLE_SIZE.
hashOne(key) = key % TABLE_SIZE,
hashTwo(key) = prime – (key%prime),
i as current index

DoubleHashing(key) = (hashOne(key) + i * hash2(key)) % TABLE_SIZE

Before, we used i for
Linear: i
Quadratic: i^2
now, we use Double: i * hash2

Heaps (Binary Tree, priority queues)

code – https://github.com/redmacdev1988/binaryHeap

Another type of Binary tree is the Heap.

A Heap is a binary tree that must have these two properties:

1) A complete binary tree is a binary tree in which every level, except possibly the last, is completely filled, and all nodes are as far left as possible.

Also, nodes are inserted from the left to right.

For example, these are complete trees:

Not complete, because because one of the leafs has a hole in it.

2) Heap property – The value of the parent node is greater than or equal to its children.

A heap is a complete binary tree that satisfies the heap property.

When the largest values start from the root, it’s called a max heap.

The opposite of that, is the min heap, where the smallest value is at the root.

Applications of Heaps

  • sorting (Heap sort)
  • graph algorithms (Shortest Path)
  • Priority Queues
  • Finding the k-th smallest or largest value

Operations of Heap

[10, 5, 17, 4, 2]

We add 10 the tree as the root.

Then add 5 from the left.

From left to right, we add, so we then add 17.

However, here 17 violates the max heap rule because parents should be larger or equal to children.

We then bubble the 17. We switch their positions and now 17 is at the root. 5 is on the left. 10 is on the right.

The heap (max heap example)

A max heap has the rule that the parent node must always have a higher priority than its child.

Each node is placed like a tree, however, take caution that there is no left/right references. Everything is accessed via index of the array. If we do want to access children of the node, its children will be indexed like so:

for the left child: 2x + 1
for the right child: 2x + 2

Inserting a node with a larger priority

Let’s say we insert a node with a higher priority than its parent. In the example below, we try to insert 12. Obviously, 12 is larger than 8, and thus, we need to readjust.

First, we check to see if its parent (index 1) is valid. It is, so we have the array reference that WAS pointing to the parent, re-point to the the new node.

Then, have the array reference (with index 3) that was originally pointing to the new node, point to the parent.

We add 4, it satisfies heap property.

We then add 22. Oops, violation.

Then then swap 22 with 5. And then swap 22 with 17.

When largest values would bubble up.

The longest a value can travel is the height of the tree.
O(log n).

height = log(n).
O(log n).

In heaps, we can only delete the root node. And then fill it in with another value that maintains the heap property.

To do this, we move the last value on the heap (5) to the root. We fix the heap properties by bubbling 5 down and moving 17 up.
We pick larger child and swap.

Time complexity is O(log n) – removing value from heap
Finding max/min value is O(1) – root

Implementation

Store everything in array. Start from root, where index is 0.

left node’s index = parent node index * 2 + 1
right node’s index = parent node index * 2 + 2

index of parent of node = Math.floor((node’s index-1)/2)

Additional Material

We have an array property in the constructor called heap. By definition, this array is contiguous memory, and can be indexed, starting from 0.

When we insert a node to the heap array, we allocate a new Node object in memory. Then the 0th index will contain a reference to that Node. When we insert the next node, another node will be allocated in memory, and we then have the 1st index reference that node.

Every node created will be inserted this way.

Thus, the latest current node will always have index heap.length – 1.

Then we add 12. However 12 violates the heap property.

Now, we recalculate the current node index and the parent node index. We want to move one level up so we can calculate index like so:

The easiest way for us to move up the tree is to simply have the current index take on the parent index. Then, recalculate the parent index via:

Math.floor((currentNodeIdx-1)/2)

As you see here, we have a violation. 12, being the child of 9, is larger than its parent. So we must bubble 12 up.

After the first bubble, we still have a violation because 12 is larger than its parent 10.
So now we just repeat what we did above.
Is the parent node valid? Yes.
Is the priority of the current node larger than the parent? Yes.

We then have the parent index’s reference point to the larger valued node. And have the current indexed element reference the smaller node. Our nodes gets reshuffled for ease of viewing, but the concept stands. Index 0 is now pointing to the node with the largest value. And all heap properties hold.

insertion code

Note that Math.floor(currentNodeIdx-1/2) is same as Math.ceil(currentNodeIdx-2/2). It gets the parent node’s index.

Heap Removal O( log n )

1) Removal means we pop the root node and process it.
2) Then we get the last element and copy it over to the root node.
3) Use top bottom to push it down if needed.

First, we copy the last value in the tree ( last element of the array ) onto the root. Then remove the last element, and decrease the heap size counter.

From there we recursively push the root value down the tree so that it satisfies the heap property of where key(parent) >= key(child) if its max heap, and <= if its min heap. We decide on which child node to go down by comparing the two child nodes. We switch the parent value with the largest of the 2 child nodes.

Full Source Code

Tests

Heap Sort

[5, 3, 10, 1, 4, 2];

add them into max Heap object.

If we keep popping…they will come out descending order.

if we use min Heap and remove them, it will come out ascending order.

shellsort (js)

Shellsort is the same as insertion sort. Except that it uses a pass array to decide how many number of steps to jump.

In Insertion Sort, we always use 1 step. When we backtrack, we go back 1 step. But in Shellsort, we have an array, where we decide how many jumps to backtrack. This is called a “gap”. Of course in the very end, we’ll definitely have 1 step at the end.

In our example, we decide to use 5, 3, 1. What this means is that we will sort this via Insertion via passes where we jump 5 steps, then 3 steps, then 1 step. So for step 5, we start at index 5 of the array. Do the Insertion sort, but instead of using 1 step when going backwards, we use 5 steps.

first pass, 5 steps

We start off at index 5. Will process the insertion sort, and when we’re done, we go down the array 1 by 1.

The insertion sort itself will use 5 steps when sorting via

Everything else is processed the same way as Insertion Sort. We get the current valueToInsert. It needs to satisfy a few points:

1) is the index larger or equal to the gap index?
2) is the previous gapped element larger than the valueToInsert?

If so, we assign the previous gapped element to the current one.
If not, we just put the valueToInsert back to the current element.

Given the Insertion Code with the gap element like so:

The gap we use is 5. So we need to do an Insertion pass where the gap is 5.

We start index 5. The value at that index is 11. 11 is the value we need to check and find a proper place to insert.

The Insertion sort happens when we have j point to i.

1) j>= gap, 5 >= 5 ? yes
2) previous gapped element > valueToInsert, 6 > 11 ? NO

dataStore[5] = 11, we move down the array.

shellsort_pass5_1

The next index is 6. dataStore[6] is 8. The previous gapped element is dataStore[6-5] is 0

1) j >= gap, 6 >= 5? yes
2) previous gapped element > valueToInsert, 0 > 8, NO

dataStore[6] = 8, we move down the array

shellsort_pass5_2

The next index is 7. dataStore[7] is 0. Its previous gapped element dataStore[7-5] is 2.

1) j >= gap, 7 >= 5? yes
2) previous gapped element > valueToInsert, 2 > 0 yes

When both requirements is true, we exchange the element values. Basically storing the previous value into the current value. This is the because the previous value is larger than the current value.

We continue on the insertion sort by looking back at -gap elements. Namely, j-= gap. Hence our j is now at index 2, dataStore[2] = 2.
We check the requirements again:

1) j >= gap, 2 >= 5 NO

So we jump out of the loop and assign the valueToInsert at dataStore[2]. This perfectly switches the numbers at dataStore[2], and dataStore[7].

shellsort_pass5_3

We move onto the next index at 8. dataStore[8] is 5.
Previous gapped element is dataStore[8-5] = 9

1) j >= gap, 8 >= 5, yes
2) previous gapped element > valueToInsert, 9 > 5 yes

Hence we need to copy the previous element to the current one. We then backtrack gap steps into index 3

1) j >= gap, 3 >= 5, NO
therefore dataStore[3] = 5.

We keep going down the array.

shellsort_pass5_4

We process the last element at index 9, dataStore[9] is 4.

dataStore[9] is 4
dataStore[9-5] is 3

1) j>= gap, 9 >= 5
2) previous gapped element > valueToInsert, 3 > 4, NO.

So we jump out of the loop
dataStore[9] = 4

next gap pass, 3 steps

We start from beginning to end, and at index gap. We do the insertion sort with gap steps, however, we process each element down the array one by one.

index 3

Hence, our next pass involves gap 3. We start off at index 3.
We see that dataStore[3] is 9. valueToInsert = 9.
Previous gap element is dataStore[0] is 6.

is index 3 >= 3 yes
6 > 9 (valeToinsert) NO, move on to next index

shellsort_pass3_1

index 4

We now on index 4.
dataStore[4] = 3, valueToInsert = 3
dataStore[4-3] = 0

4 >= 3 yes
0 > 3 (valeToInsert) NO, move on to next index

index 5

We now on index 5.
dataStore[5] is 11, valueToInsert = 11
dataStore[5-3] is 2

5 >= 3 yes
2 > 11 (valeToInsert), NO, move on to next index

shellsort_pass3_2

index 6

We now on index 6.
dataStore[6] is 8 valueToInsert = 8
dataStore[6-3] is 9.

6 >= 3 yes
9 > 8(valeToInsert), we then copy over the value from the previous element to the current

Thus, dataStore[6] = 9.

We go back 1 gap step so now j is at index 3.
dataStore[3] is 9
dataStore[3-3] is 6

3 >= 3 yes
6 > 8 (valeToInsert), NO, we skip out. have current index j, dataStore[3] = 8 (valueToInsert)

move forward down the array

shellsort_pass3_3

index 7

We are now at index 7.
dataStore[7] is 0. valueToInsert is 0.
dataStore[7-3] is 3

7>=3 yes
3 > 0 yes

Let’s copy previous element to current so now
dataStore[7] is 3.

j moves a “gap” step back to index 4

dataStore[4] is 3
dataStore[1] is 0

4 >= 3, YES
0 > 0 NO

dataStore[4] is 0.

Move forward

shellsort_pass3_4

index 8

dataStore[8] is 5 (valueToInsert)
dataStore[8-3] is 11

8 >= 3 yes
11 > 5 yes

So we need to copy the previous value over to current.

dataStore[8] = 11

Go back 1 “gap” step to index 5

dataStore[5] is 11
dataStore[5-3] is 2

5 >= 3 yes
2 >= 5 (valueToInsert) NO

dataStore[5] = 5

move on…

shellsort_pass3_5

Index 9

Finally, we are index 9.

dataStore[9] is 4 (valueToInsert)
dataStore[9-3] is 8

9 >= 3 yes
8 > 4 yes

so we copy the previous gap element to current
dataStore[9] = 8

we go previous gap element to index 6

dataStore[6] is 8
dataStore[6-3] is 9

6 >= 3, YES
9 >= 4 (valueToInsert), YES

copy the previous gap element to current
dataStore[6] = 9

go back “gap” elements

dataStore[3] is 9
dataStore[0] is 6

3 >= 3, YES
6 > 4 (valueToInsert) YES

copy the previous gap element to current
dataStore[3] = 6

go back “gap” elements

index is now 0
0 >= 3 NO

so dataStore[0] = 4 (valueToInsert)

Done for gap 3 pass

shellsort_pass3_6

Full Source

Circular List in JS (prototype pattern, constructor pattern)

Prototype Pattern

– Using prototypes to create classes, all instances of that class will the same functions.

– Can’t have private utility functions. All properties and functionalities can be seen by caller.

Constructor Pattern with scoped private variables

By using scoped variables, we can have privacy, and can create get/set functionalities for public access.

However, not very practical as I will need to create get/set functions for all the private variables.
Then, using them in my functions can be awkward as assigning references must be dealt with using set functions.
And being assigned, we’ll have to call the get functions.

Insertion Sort

http://www.geeksforgeeks.org/insertion-sort/

Insertion sort is a simple sorting algorithm that works the way we sort playing cards in our hands

Say we’re given a data set, 8 5 3 12 2

Option 1 – previous is larger

We start off the algorithm at index 1.
So we see 5 and store it in a temp variable.

There are 2 conditions we must satisfy:

1) Is the current index > 0 ? (1 > 0 ? yes )
2) Is the previous element is larger than our temp? 8 > 5, yes.

Both satisfies so we copy the previous element (8) to the current location where 5 is.

We move 1 step back and hit index 0.

We evaluate the 2 situations again:

1) 0 > 0, no. We stop.

If we reach index 0, that means we’ve reached the end, and we store 5 there.

We then move on to index 2 and continue processing in the same manner.

Option 2 – previous is smaller

If say we have 3 5 7.

We start off at index 1.
We see 5 and store it in a temp variable.

There are 2 conditions we must satisfy:

1) Is the current index > 0 ? (1 > 0 ? yes )
2) Is the previous element is larger than our temp? 3 > 5, no.

In this case, we don’t do anything move onto the next index 2.

This is the part where if the whole array is sorted, we only execute this comparison and then move on to the next index. Since the previous will always be smaller than the current, we simply stop and move on. Hence, the best case is O(n) because we do execute a comparison at every index once. We do this for every element and then stop.

Insertion sort takes the current number and go backwards to see where to insert it. We do this by comparing the previous element and seeing if its bigger than our valueToInsert. If it’s larger, push it down and we keep going. If it’s smaller, we stop and we insert our number. If we reach the end at index 0, we insert our number.

insertionsort_ex_a

Now we have 5 8 3 12 2.

We work our next pass which is index 2 with value 3.

We store 3 and start searching for the insertion point. We evaluate the 2 points:

1) is index > 0? 2 > 0, yes.
2) We look at the previous element 8. is 8 > 3? yes, so we move 8 to where 3 is.

Move backwards 1 step.

1) is index > 0? 1 > 0, yes.
2) Then we look at previous element 5, is 5 > 3? yes. so we move 5 to where 8 is.

1) is index > 0? 0 > 0, no. We’ve reached the end. Hence we put 3 at index 0.

This leaves us with 3 5 8 12 2.

insertionsort_ex_b

Then we work on value 12.

1) is index > 0? index 3 > 0, yes.
2) Previous element 8 > 12?
no. We stop, and move on.

Moving on…finally, our last value is 2. Apply the same as we did above and we’ll see that 2 will be inserted at the way in the beginning because it is smaller than the other numbers.

insertionsort_ex_c

full source

Time Complexity

The best case input is an array that is already sorted. In this case insertion sort has a linear running time (i.e., O(n)). During each iteration, the first remaining element of the input is only compared with the right-most element of the sorted subsection of the array.

The simplest worst case input is an array sorted in reverse order. The set of all worst case inputs consists of all arrays where each element is the smallest or second-smallest of the elements before it. In these cases every iteration of the inner loop will scan and shift the entire sorted subsection of the array before inserting the next element. This gives insertion sort a quadratic running time O(n^2).

The average case is also quadratic, which makes insertion sort impractical for sorting large arrays.

However, insertion sort is one of the fastest algorithms for sorting very small arrays, even faster than quicksort; indeed, good quicksort implementations use insertion sort for arrays smaller than a certain threshold, also when arising as subproblems; the exact threshold must be determined experimentally and depends on the machine, but is commonly around ten.

Selection sort

https://en.wikipedia.org/wiki/Selection_sort

Worst-case performance О(n2) comparisons, О(n) swaps
Best-case performance О(n2) comparisons, О(n) swaps

The idea is that we latch onto the first element and deem it being the minimum.
We then go through the rest of the array, and look for anything that is smaller. If so, we swap. This is 1 pass.

At this point, after the 1st pass, we deem first element as minimum. Then we move on
to the next element and call it the minimum. We do another pass and look in the leftover array
to see if there’s anything smaller than our minimum. If so, swap.

We keep doing this, pulling minimum numbers from the array and pushing it towards the front, until
all the elements are sorted from smallest to largest.

selectionsort-outer-inner

So there’s 2 loops: outer and inner.

Outer simply goes through the array.

The inner goes through the remaining elements AFTER the outer via a loop and compares it to the outer.

if the inner element is smaller than the outer element, then we swap. Essentially what is happening is that we are bringing in the smallest element to the front.

Example

Given an array, we make passes on it where:

– we select the starting value to be the minimum.
– we then make (length of array – 1) passes over it. (through all of our passes, the last element will be automatically sorted, so no need to do anything on it)
– For every pass, we need to do the very important action of pulling the minimum from the array and swapping it with the first value.

Example

…continuing on in the same fashion…

Eventually all the smallest number will be swapped to the front, resulting in the list being sorted.

full source

Stack and Heap

https://stackoverflow.com/questions/79923/what-and-where-are-the-stack-and-heap?rq=1

The stack is the memory set aside as scratch space for a thread of execution.

When a function is called, a block is reserved on the top of the stack for local variables, parameters, and some bookkeeping data.

When that function returns (finishes execution), the block becomes unused (all the variables gets popped) and can be used the next time a function is called.

The stack is always reserved in a LIFO (last in first out) order; the most recently reserved block is always the next block to be freed. This makes it really simple to keep track of the stack; freeing a block from the stack is nothing more than adjusting one pointer.

The heap is memory set aside for dynamic allocation. Unlike the stack, there’s no enforced pattern to the allocation and deallocation of blocks from the heap; you can allocate a block at any time and free it at any time. This makes it much more complex to keep track of which parts of the heap are allocated or free at any given time; there are many custom heap allocators available to tune heap performance for different usage patterns.

Each thread gets a stack, while there’s typically only one heap for the application (although it isn’t uncommon to have multiple heaps for different types of allocation).

The stack is attached to a thread, so when the thread exits the stack is reclaimed. The heap is typically allocated at application startup by the runtime, and is reclaimed when the application (technically process) exits.

The size of the stack is set when a thread is created. The size of the heap is set on application startup, but can grow as space is needed (the allocator requests more memory from the operating system).

The stack is faster because the access pattern makes it trivial to allocate and deallocate memory from it (a pointer/integer is simply incremented or decremented), while the heap has much more complex bookkeeping involved in an allocation or deallocation. Also, each byte in the stack tends to be reused very frequently which means it tends to be mapped to the processor’s cache, making it very fast.

Another performance hit for the heap is that the heap, being mostly a global resource, typically has to be multi-threading safe, i.e. each allocation and deallocation needs to be – typically – synchronized with “all” other heap accesses in the program.