11) Introduction to Parallel Scaling

11) Introduction to Parallel Scaling#

Las- Intro to Multithreading

Today:

  1. Programs with more than one part

1. Programs with more than one part#

So far, we’ve focused on simple programs with only one part, but real programs have several different parts, often with data dependencies.

Some parts will be amenable to optimization and/or parallelism and others will not.

Diminishing returns

This principle is called Amdahl’s Law, which is a formula that shows how much faster a task can be completed when more resources are added to the system.

function exec_time(f, p; n=10, latency=1)
    # Suppose that a fraction f of the total work is amenable to optimization
    # We run a problem size n with parallelization factor p
    return latency + (1-f)*n .+ f*n./p
end

exec_time (generic function with 1 method)

Let’s see a few fractions for example:

using Plots
using DataFrames
using Printf
default(linewidth=4, legendfontsize=12)

ps = exp10.(range(log10(1), log10(1000), length=50))

plot(ps, [exec_time(.99, ps, latency=0), exec_time(1, ps, latency=0)],
    xscale=:log10, yscale=:log10, labels=["f=0.99" "f=1"],
    title="Strong scaling", xlabel="p", ylabel="time")
Contents