Empirical Analysis

An empirical approach analyzes the performance of an algorithm by setting up a controlled environment and measuring the execution time. The graph shows the comparison of different sorting algorithms as a function of the input size.

int multiplyRemainderByIntegerPart(int i, int j) {
  int remainder = i % j;
  int integerPart = i / j;
  return remainder * integerPart;
}

The empirical approach is useful but has limitations, such as the high cost of implementation and dependence on the hardware. Therefore, asymptotic analysis is preferred, as it is independent of the hardware and covers a broader range of inputs.

Asymptotic Analysis

Asymptotic analysis focuses on the behavior of algorithms for large inputs. It is based on the assumption that the cost of primitive operations is constant (O(1)).

Example: Algorithm with Constant Cost

int multiplyRemainderByIntegerPart(int i, int j) {
  int remainder = i % j;
  int integerPart = i / j;
  return remainder * integerPart;
}

In this example, the execution time is constant, regardless of the input size. For more complex algorithms, such as search or sorting, the execution time may vary depending on the number of iterations or elements in the array.

Worst Case

When there are loops or conditionals, analyzing the worst case is important to ensure the algorithm behaves as expected in extreme scenarios.

public static boolean contains(int[] v, int n) {
  for (int i = 0; i < v.length; i++) {
    if (v[i] == n)
      return true;
  }
  return false;
}

For algorithms that involve iteration or nested loops, the execution time may be linear or quadratic, depending on the number of elements and the depth of the loops.

Asymptotic analysis helps us simplify these functions to understand the behavior of algorithms with large inputs, representing them through notations like O(n), O(log n), or O(n²), making it easier to compare and choose the best solution.

Algorithm analysis, both empirical and asymptotic, is essential to ensure the efficiency and performance of code, especially in scenarios involving large volumes of data or when performance is critical.

1. Brute Force

This technique explores all possible options to find a solution. While simple to implement, it's often inefficient.

Example: Find a pair with a given sum

function hasPairWithSum(arr, sum) {
  for (let i = 0; i < arr.length; i++) {
    for (let j = i + 1; j < arr.length; j++) {
      if (arr[i] + arr[j] === sum) return true;
    }
  }
  return false;
}

2. Greedy Algorithms

Greedy methods build a solution step by step, always choosing the local optimum. This works for problems where local choices lead to a global solution.

Example: Coin Change (Greedy)

Choose the largest coin possible at each step. This works for standard denominations but may fail in arbitrary systems.

3. Divide and Conquer

This technique splits the problem into smaller subproblems, solves them recursively, and combines the results.

Example: Merge Sort

function mergeSort(arr) {
  if (arr.length <= 1) return arr;
  const mid = Math.floor(arr.length / 2);
  const left = mergeSort(arr.slice(0, mid));
  const right = mergeSort(arr.slice(mid));
  return merge(left, right);
}

4. Dynamic Programming (DP)

DP solves problems by breaking them into overlapping subproblems and storing the results of subproblems to avoid redundant work.

Example: Fibonacci with Memoization

function fib(n, memo = {}) {
  if (n <= 1) return n;
  if (memo[n]) return memo[n];
  memo[n] = fib(n - 1, memo) + fib(n - 2, memo);
  return memo[n];
}

Final Thoughts

Understanding these core techniques is essential for improving your problem-solving skills. Over time, you'll learn to recognize which strategy applies best to different problem types — and how to combine them for more complex scenarios.

How Greedy Algorithms Work

A greedy algorithm follows this simple strategy:

1. Make the best choice at the current step.
2. Move to the next step without revisiting previous decisions.
3. Repeat until the problem is solved.

Characteristics of Greedy Algorithms

• Greedy Choice Property: A global optimum can be arrived at by choosing the local optimum.
• Optimal Substructure: A problem has an optimal substructure if an optimal solution can be built from optimal solutions of its subproblems.

Not all problems have these properties. If they do, a greedy algorithm can be very efficient.

Example: Activity Selection

Given n activities with start and end times, select the maximum number of activities that don't overlap.

import java.util.*;

class Activity {
  int start, end;

  Activity(int start, int end) {
    this.start = start;
    this.end = end;
  }
}

public class GreedyActivitySelector {
  public static void main(String[] args) {
    List<Activity> activities = List.of(
      new Activity(1, 3),
      new Activity(2, 5),
      new Activity(4, 7),
      new Activity(6, 9)
    );

    activities.sort(Comparator.comparingInt(a -> a.end));

    int lastEnd = 0;
    for (Activity a : activities) {
      if (a.start >= lastEnd) {
        System.out.println("Selected: " + a.start + "-" + a.end);
        lastEnd = a.end;
      }
    }
  }
}

This is a classic example of greedy algorithm success.

Common Greedy Algorithm Problems

• Activity Selection Problem
• Fractional Knapsack Problem
• Huffman Encoding
• Prim’s and Kruskal’s algorithms (Minimum Spanning Tree)
• Dijkstra’s algorithm (Shortest Path)

When Greedy Fails

Greedy algorithms do not always produce the optimal solution. The 0/1 Knapsack Problem is one such case, where dynamic programming gives the correct answer, but greedy fails.

Greedy algorithms offer simple, intuitive solutions with great performance for many problems. Understanding their limitations and identifying problems that fit their structure is key to using them effectively.

When to Use Dynamic Programming

Dynamic Programming is ideal when:

• The problem can be divided into overlapping subproblems.
• The problem has an optimal substructure (the optimal solution can be built from optimal solutions of subproblems).

Memoization (Top-Down Approach)

Memoization is an optimization technique where you store the results of expensive function calls and return the cached result when the same inputs occur again.

Example: Fibonacci with Memoization

import java.util.HashMap;

public class Fibonacci {
  private HashMap<Integer, Integer> memo = new HashMap<>();

  public int fib(int n) {
    if (n <= 1) return n;
    if (memo.containsKey(n)) return memo.get(n);
    int result = fib(n - 1) + fib(n - 2);
    memo.put(n, result);
    return result;
  }
}

This approach reduces the exponential time complexity of the naive recursive Fibonacci to linear time.

Tabulation (Bottom-Up Approach)

Tabulation builds up the solution iteratively using a table (usually an array). It avoids recursion entirely.

Example: Fibonacci with Tabulation

public int fib(int n) {
  if (n <= 1) return n;
  int[] dp = new int[n + 1];
  dp[0] = 0;
  dp[1] = 1;
  for (int i = 2; i <= n; i++) {
    dp[i] = dp[i - 1] + dp[i - 2];
  }
  return dp[n];
}

Tabulation is usually more efficient in terms of space and avoids the overhead of function calls.

Choosing Between Memoization and Tabulation

• Use memoization if the recursive structure of the problem is intuitive and you prefer a top-down style.
• Use tabulation if you want better performance and can write the solution in a bottom-up way.

Common DP Problems

• Fibonacci sequence
• Longest Common Subsequence (LCS)
• 0/1 Knapsack Problem
• Coin Change
• Matrix Chain Multiplication
• Edit Distance

Dynamic Programming is a key technique in algorithm design. Mastering both memoization and tabulation will significantly improve your ability to solve complex problems efficiently.

What is Recursion?

A recursive function is defined in terms of itself. Every recursive solution must have:

• Base Case – A condition under which the function returns without calling itself.
• Recursive Case – The part where the function calls itself to solve a subproblem.

Example: Factorial

public int factorial(int n) {
  if (n == 0 || n == 1)
    return 1;
  return n * factorial(n - 1);
}

In this Java example, factorial(5) calls factorial(4), which calls factorial(3), and so on, until it reaches the base case.

Stack Behavior

Recursion uses the call stack. Each function call is placed on the stack and waits until deeper calls return.

Poorly written recursive functions may cause stack overflow if the base case is missing or unreachable.

When to Use Recursion

Recursion is ideal when:

• The problem can be divided into similar subproblems.
• You’re working with hierarchical data (like trees).
• Iterative solutions are complex or less readable.

Common Recursive Problems

• Fibonacci sequence
• Tree traversal (preorder, inorder, postorder)
• Backtracking problems (e.g., Sudoku, N-Queens)
• Divide and conquer algorithms (e.g., MergeSort, QuickSort)

Example: Binary Search (Recursive)

public int binarySearch(int[] arr, int target, int low, int high) {
  if (low > high) return -1;
  int mid = low + (high - low) / 2;
  if (arr[mid] == target) return mid;
  else if (arr[mid] > target)
    return binarySearch(arr, target, low, mid - 1);
  else
    return binarySearch(arr, target, mid + 1, high);
}

This recursive binary search breaks the array into smaller pieces at each call.

Pros and Cons

Pros:

• Elegant and concise code.
• Great for naturally recursive problems.

Cons:

• Risk of stack overflow.
• Can be less efficient than iteration due to function call overhead.

Recursion is a core concept that every developer should master. Once you understand how it works and when it’s appropriate, it becomes a valuable tool in your problem-solving toolkit.

Binary Search Example

function binarySearch(arr, target) {
  let left = 0, right = arr.length - 1;
  while (left <= right) {
    const mid = Math.floor((left + right) / 2);
    if (arr[mid] === target) return mid;
    if (arr[mid] < target) left = mid + 1;
    else right = mid - 1;
  }
  return -1;
}

At each step, the search space is divided by 2. This results in log₂(n) steps in the worst case, making it significantly faster than linear search for large datasets.

When to Expect Logarithmic Complexity

Algorithms that:

• Divide the problem space repeatedly
• Work with sorted data or tree-like structures (e.g., binary search trees, heaps)
• Use recursive halving strategies

These typically fall under O(log n) time.

Why It Matters

Logarithmic algorithms scale extremely well. Even for large input sizes, the number of steps grows slowly. For example:

• n = 1,000 → log₂(n) ≈ 10
• n = 1,000,000 → log₂(n) ≈ 20

Understanding and identifying opportunities to use O(log n) algorithms is a major step toward writing highly efficient code.

What is a Data Structure?

A data structure is a way of organizing and storing data so that it can be accessed and modified efficiently. Different types of data structures are suited for different kinds of applications.

Common Types

• Array – A collection of elements stored in contiguous memory.
• Linked List – A linear collection of elements, where each element points to the next.
• Stack – A LIFO (Last-In-First-Out) structure.
• Queue – A FIFO (First-In-First-Out) structure.
• Hash Table – Stores key-value pairs and provides efficient lookup.
• Tree – A hierarchical structure with nodes.
• Graph – A collection of nodes connected by edges.

Choosing the right data structure impacts both memory usage and execution time.

What is an Algorithm?

An algorithm is a finite sequence of well-defined steps for solving a problem or performing a computation.

Common Algorithm Categories

• Sorting – Organizing data (e.g., QuickSort, MergeSort)
• Searching – Finding items in a data structure (e.g., Binary Search)
• Graph Traversal – Navigating graph data (e.g., DFS, BFS)
• Dynamic Programming – Solving problems using memoization and recursion.
• Greedy Algorithms – Making locally optimal choices at each step.

Relationship Between Data Structures and Algorithms

The efficiency of an algorithm often depends on the underlying data structure. For example:

• Binary Search requires a sorted array.
• Dijkstra’s Algorithm uses a priority queue (often implemented with a heap).
• DFS/BFS work with stacks or queues, respectively.

The right combination of structure and algorithm ensures optimal performance.

Real-World Example

// Using a HashMap to count word frequency
public Map<String, Integer> countWords(String[] words) {
  Map<String, Integer> map = new HashMap<>();
  for (String word : words) {
    map.put(word, map.getOrDefault(word, 0) + 1);
  }
  return map;
}

In this Java example, a hash map is used to efficiently count word frequencies in an array—demonstrating how the choice of data structure directly affects algorithm performance.

Why Learn DSA?

Understanding DSA enables developers to:

• Solve complex problems more efficiently.
• Perform well in technical interviews.
• Optimize code for speed and memory.
• Understand the trade-offs of different solutions.

Whether you’re working on a small app or a large-scale system, mastering DSA is a critical skill in software engineering.

Big-O Notation (O)

Big-O describes the upper bound — the worst-case scenario. It gives us a way to guarantee that an algorithm won't perform worse than a certain threshold.

function linearSearch(arr, target) {
  for (let i = 0; i < arr.length; i++) {
    if (arr[i] === target) return i;
  }
  return -1;
}

In the worst case, the target is at the end of the array or not present. The function runs through the entire array, so the time complexity is O(n).

Big-Ω Notation (Ω)

Big-Ω gives us the lower bound — the best-case scenario. It tells us how well an algorithm can perform in ideal conditions.

Using the same linearSearch example, if the target is the first element, the loop ends immediately. In this case, the time complexity is Ω(1), meaning it completes in constant time.

Big-Θ Notation (Θ)

Big-Θ describes the tight bound — when an algorithm's best and worst cases grow at the same rate. If an algorithm is always linear regardless of input conditions, we say it is Θ(n).

This notation is less commonly used in isolation, but it’s helpful when you want a complete picture of an algorithm's growth.

Why This Matters

Time complexity helps you:

• Predict how an algorithm scales
• Choose between alternative solutions
• Spot inefficient code before performance becomes a problem

Understanding how to apply these notations is a key skill in writing scalable, efficient software. When designing algorithms, always consider how performance changes as inputs grow — and aim to choose or design solutions with the best time complexity you can afford.