apnorton

Downloading an HLS Stream with ffmpeg

Mon, 19 Jun 2023 17:43:01 -0400

I recently encountered a video served with an HLS stream that I wanted to have a copy of, but the website did not support downloading it. Trying yt-dlp yielded no results (it was not one of the “common” websites), but I did learn that ffmpeg supports conversion of HLS streams directly to a file. I referenced a blog post on codementor.io, though I’m sure that the same information is present in many other locations online.

Meeting Don Knuth

Fri, 07 Dec 2018 12:00:00 +0000

Everyone has heroes, but not everyone gets to meet them. This past Tuesday, I was fortunate to meet one of mine — Don Knuth. If you are in computer science and don’t know who Don Knuth is, I highly recommend you take a break from this post and do a bit of reading about him. He is most well-known for authoring his (still in progress) Art of Computer Programming, a compendium of all kinds of information on algorithms, but his contributions to computer science and software development reach far beyond that. He invented TeX (precursor to LaTeX), wrote Concrete Mathematics (a great mathematical foundations of computer science book), and received a Turing award.

Me with Don Knuth

Raspberry Pi Default Groups

Sat, 25 Nov 2017 13:54:28 +0000

In setting up my Raspberry Pi for a home fileshare, I noticed the pi user is a part of several default groups. These are:

1pi adm dialout cdrom sudo audio video plugdev games users input netdev gpio i2c spi

(I’m using the 2017-09-07 image of Raspbian Stretch Lite.)

This looked like a lot of groups to me! To make sure my new user only has the minimum permissions needed, let’s look at the what each group is and why it’s there.

How to Request a Regrade

Tue, 04 Jul 2017 17:02:25 +0000

One of the highlights of my time at UVA was working as a teaching assistant for the computer science department. In this capacity, I proctored labs and exams, held office hours, created exam questions, and even graded homework and exams. Due, in part, to the large class sizes of our introductory courses and the necessity of multiple graders for each assignment, most professors have a “regrade” policy – if the grader has made an error in grading a student’s work, there is a formal process for the student to request a second look at his or her work.

For CS 2150 (the course I TA’d), I was one of two or three TAs who processed most – if not all – of the regrades for exams in the past two semesters. Although grades are ideally determined solely by the answer’s merit, there are a few simple ways you can make your grader’s life easier. (And that’s always a good thing, right?)

A Brief Exploration of a Möbius Transformation

Tue, 18 Apr 2017 20:05:42 +0000

As part of a recent homework set in my complex analysis course, I was tasked with a problem that required a slight generalization on part of Schwarz’s Lemma:

Lemma (Schwarz): Let $f$ be analytic on the unit disk with $|f(z)| \leq 1$ for all $z$ on the disk and $f(0) = 0$. Then $|f(z)| < |z|$ and $f’(0)\leq 1$.
If either $|f(z)|=|z|$ for some $z\neq0$ or if $|f’(0)|=1$, then $f$ is a rotation, i.e., $f(z)=az$ for some complex constant $a$ with $|a|=1$.

The homework assignment asked us (within the context of a larger problem) to consider the case when $f(\zeta) = 0$ for some $\zeta \neq 0$ on the interior of the unit disk. The secret to this problem was to find some analytic function $\varphi$ that maps the unit disk to itself, but swaps $0$ and $\zeta$. Then, we may consider $\varphi^{-1}\circ f\circ \varphi$ and apply Schwarz’s Lemma.

How I wrote a GroupMe Chatbot in 24 hours

Tue, 28 Feb 2017 23:05:39 +0000

For the past couple years, I have worked as a teaching assistant for UVa’s CS 2150 (Program and Data Representation) course. We recently started a GroupMe chat for the course staff, and I thought it would be fun to create a chatbot to help remind all the TAs to submit timesheets, keep track of when people are holding office hours, and remember when/where TA meetings are being held. Setting up a basic chatbot is a lot simpler than it sounds and is really fun–I wrote my bot from scratch using Python in just one day.

The 2150 chatbot

GroupMe Bot Overview

GroupMe has a very brief tutorial explaining how their API may be used for bots. The easiest way to create a bot is through their web form, which prompts you for the bot’s name, callback URL (technically optional, but you want it), avatar URL (optional), and the name of the group where the bot will live. Once you’ve done this, you will be given a unique bot ID token. Anyone with this token can pretend to be your bot, so keep it safe. (Security is somewhat laughable here: your bot asserts its ID and the server believes it with no “login” procedure.) We’ll talk more about the callback URL in a moment; for now, just leave it blank.

Once you’ve done these steps, you have created a bot–as far as GroupMe is concerned. If you send a specifically formatted JSON mssage, the newly created bot will post in your group. However, if left at this point, your “bot” is little more than a proxy for human-written messages submitted with curl. Your bot needs some way of reading messages sent to the group, formulating a response, and only then sending messages to the GroupMe servers.

TensorFlow with the Surface Book

Wed, 04 Jan 2017 15:32:45 +0000

While interning at Microsoft over the summer, I received a first-generation Surface Book with an i5-6300U CPU (2.4 GHz dual core with up to 3.0 GHz), 8GB RAM, and a “GeForce GPU” (officially unnamed, but believed to be equivalent to a GT 940). This is a huge step up from my older laptop, so I wanted to set it up for my ML work. In this post, I’ll outline how I set it up with TensorFlow and GPU acceleration.

CUDA + cuDNN

If you want to use GPU acceleration, the typical way to do so is with NVIDIA’s CUDA API. CUDA 8.0 is compatible with the Surface Book and is (as of this writing) the most up-to-date version of CUDA. Download it from the NVIDIA website and run their installer.

For work with deep learning, you’ll also want to install cuDNN. To install, just download the library from NVIDIA’s website and unzip it in a convenient place (I chose C:\cudnn). The only “installation” you need to do is to add C:\cudnn\bin to your PATH environment variable.

Visualizing Multidimensional Data in Python

Mon, 19 Dec 2016 20:51:55 +0000

Nearly everyone is familiar with two-dimensional plots, and most college students in the hard sciences are familiar with three dimensional plots. However, modern datasets are rarely two- or three-dimensional. In machine learning, it is commonplace to have dozens if not hundreds of dimensions, and even human-generated datasets can have a dozen or so dimensions. At the same time, visualization is an important first step in working with data. In this blog entry, I’ll explore how we can use Python to work with n-dimensional data, where $n\geq 4$.

Packages

I’m going to assume we have the numpy, pandas, matplotlib, and sklearn packages installed for Python. In particular, the components I will use are as below:

1import matplotlib.pyplot as plt
2import pandas as pd
3
4from sklearn.decomposition import PCA as sklearnPCA
5from sklearn.discriminant_analysis import LinearDiscriminantAnalysis as LDA
6from sklearn.datasets.samples_generator import make_blobs
7
8from pandas.tools.plotting import parallel_coordinates

Plotting 2D Data

Before dealing with multidimensional data, let’s see how a scatter plot works with two-dimensional data in Python.

First, we’ll generate some random 2D data using sklearn.samples_generator.make_blobs. We’ll create three classes of points and plot each class in a different color. After running the following code, we have datapoints in X, while classifications are in y.

1X, y = make_blobs(n_samples=200, centers=3, n_features=2, random_state=0)

To create a 2D scatter plot, we simply use the scatter function from matplotlib. Since we want each class to be a separate color, we use the c parameter to set the datapoint color according to the y (class) vector.

A Microsoft Summer, Part 1: Seattle Fun

Fri, 30 Sep 2016 01:26:20 +0000

As suggested by this post’s title, I spent this past summer as an intern with Microsoft in Redmond, Washington. The experience was highly educational for me–as my first (and last!) “real” internship, I learned a lot about software development and the importance of corporate culture, as well as discovering a lot about myself. Overall, the experience was a positive one, though, and I had an enormous amount of fun!

This is the first of a three-part series on my time at Microsoft. This post focuses on fun recreational activities for interns in the Seattle area.

Outdoors

Hiking in the North Cascades

The Pacific Northwest is home to some of the most amazing views I’ve ever seen. Seattle is conveniently located close to the beach, the mountains, Puget Sound, rainforests, and many hiking trails and campsites. Exploring the outdoors also has the advantage of being very inexpensive, which is great if you’re saving your internship money for college expenses. If you visit National Parks, consider the National Park Passport Program–if you’re going to once-in-a-lifetime parks, it’s a good idea to get your passbook stamped!

Ada's Technical Books

Sun, 07 Aug 2016 01:01:24 +0000

This summer, I’ve been working as an intern for Microsoft on the Direct2D/DirectWrite team. While I can’t really talk about what my work entails, I can talk about some of the fun things I’ve done this summer in my free time and the non-work-related components of my internship. I suppose most people wouldn’t start blogging about their internship by describing a bookstore, but I went to this place today and it was so incredible that I had to write about it.

In Capitol Hill, there’s a small store by the name Ada’s Technical Books. It’s in a house that’s been converted to a cafe and bookstore, and is quite possibly the most amazing bookstore I’ve ever seen. As you walk in, you’re greeted by an small cafe counter to your left and an open area to your right with short bookcases and comfy chairs. Toys, puzzles, and “Maker”-appropriate items like lockpicks and Raspberry Pis.

Who can resist a giant 555 timer?

Visualizing Graphs in Program Output

Tue, 08 Mar 2016 17:11:35 +0000

Many computer science problems utilize graph-based data structures. Their use can range from explicit inclusion in an algorithm-centric problem (like path-finding) to a more “behind-the-scenes” presence in Bayesian networks or descriptions of finite automata. Unfortunately, visualizing large graphs can be difficult to do, especially for debugging. Unlike lists or dictionaries, which can be represented clearly by plain text printing, depicting a graph tends to require more graphics overhead than is reasonable for most programmers to write simply for debugging purposes. I’ve found that Graphviz, a free graph visualization utility, can be quite useful in debugging graph-related programs.

Installing Graphviz

If you’re on a Debian-based Linux OS (e.g. Ubuntu), you can install Graphviz using apt-get. Just run $ sudo apt-get install graphviz and you’ll have everything you need to complete the steps in this blog post. Mac OS X users can use brew equivalently.

Windows users should install using a binary downloaded from the Graphviz Windows page, but there might be some issues with setting the PATH variable for running in the commandline.

Making a basic graph

Once you’ve installed, the next thing you’ll want to do is create a basic graph to ensure the installation succeeded and to gain practice using Graphviz tools. We do this by creating a *.dot file that describes the graph we wish to display. If you’re the type of person who likes to jump right in and experiment first before reading too much, or if you love formal language specification, the DOT grammar is fairly readable and can give a quick introduction to creating DOT files.

The below is a fairly representative DOT file to demonstrate some of the capabilities of Graphviz. Open your favorite text editor, copy/paste it in, and save it as firstgraph.dot:

 1digraph G {
 2 // Style information for nodes
 3 A [style="filled", color=".05 .3 1.0"]
 4
 5 // Edge declarations
 6 A -> {B, C};
 7 D -> E [label="-1"];
 8 E -> F [label="3"];
 9 F -> D [label="10"];
10}

This creates a directed graph (also called a digraph) with six nodes and two connected components. Some of the edges have labels, and one of the nodes is colored. After you’ve copied (or downloaded) this file, open up a terminal to the directory with firstgraph.dot in it and run $ dot firstgraph.dot -Tpng -o firstgraph.png. The resulting image file should look something like the below:

The rendered `firstgraph.dot`

Deranged Exams: An ICPC Problem

Thu, 15 Oct 2015 20:24:00 +0000

This past week, my ICPC team worked the 2013 Greater New York Regional problem packet. One of my favorite problems in this set was Problem E: Deranged Exams. The code required to solve this problem isn’t that complicated, but the math behind it is a little unusual. In this post, I aim to explain the math and provide a solution to this problem.

Problem Description

The full problem statement is archived online; in shortened form, we can consider the problem to be:

Given a “matching” test of $n$ questions (each question maps to exactly one answer, and no two questions have the same answer), how many possible ways are there to answer at least the first $k$ questions wrong?

It turns out that there’s a really nice solution to this problem using a topic from combinatorics called “derangements.” (Note that the problem title was a not-so-subtle hint towards the solution.)

Derangements

While the idea of a permutation should be familiar to most readers, the closely related topic of a derangement is rarely discussed in most undergraduate curriculum. So, it is reasonable to start with a definition:

A derangement is a permutation in which no element is in its original place. The number of derangements on $n$ elements is denoted $D_n$; this is also called the subfactorial of $n$, denoted $!n$.

The sequence $\langle D_n\rangle$ is A000166 in OEIS (a website with which, by the way, every competitive programmer should familiarize themselves).

It turns out that there is both a recursive and an explicit formula for $D_n$:

{% math %} \begin{aligned} D_n &= (-1)^n \sum_k\binom{n}{k} (-1)^k k! \ &= n\cdot D_{n-1} + (-1)^n;;(D_0=1) \end{aligned} {% endmath %}

This is significant because we can use the explicit formulation for computing single values of derangements, or we can use dynamic programming to rapidly compute $D_n$ for relatively small $n$.

Eight Books on Math and Computer Science

Tue, 09 Jun 2015 00:41:23 +0000

A friend recently emailed me asking for titles of books I’d recommend to read over the summer, particularly to prepare for computer science and mathematics. I’ve adapted my suggestions into this post. I’d like to note that I’ve restricted my responses to “non-textbooks;” otherwise, I’d have several more additions that would increase the average page count and price quite drastically. As such, these books don’t have problems to work or present an extreme level of detail, but in many cases they present enough information to provide a strong foundation and context for math and CS classes.

From Mathematics to Generic Programming

Alexander Stepanov and Daniel Rose (on Amazon)

I will most likely write a separate blog post about this book. I read it during the end of the fall semester and found that it presented a very interesting approach to designing reusable code by utilizing principles from abstract algebra. It’s written to be accessible by someone who hasn’t studied abstract algebra yet, which means it also can serve as an introduction to that subject.

CODE: The Hidden Language of Computer Hardware and Software

Charles Petzold (on Amazon)

Four years ago, I wrote a review of this book on RoboDesigners. At that time, my perspective was that of a high school student and I thought the book was interesting; with the additional perspective of a year of college study in Computer Science, I cannot recommend this book highly enough.

By “building” a computer piece-by-piece from the idea of a relay through developing a simple assembly language, it covers nearly all of the material as the Digital Logic Design course I took, but in an easy-to-read book. If you comprehend the material in this book, you will be able to coast through DLD.

A Mathematician’s Apology

G. H. Hardy (on Amazon)

When a mathematician with Hardy’s stature writes a book on why he studies math, it’s probably advisable to read it! Multiple professors of mine have said it’s a book any mathematician should read and I wholeheartedly agree. It’s really short (the printing I’ve linked above is only 154 pages), but the content is amazing. Hardy addresses the complaints many have with pure math and embodies the spirit of “doing mathematics for mathematics’ sake.” If you are thinking about pursuing a theoretical route in either CS or math, I highly recommend you read this book.

How to Learn Haskell

Mon, 14 Jul 2014 20:55:08 +0000

To grow my programming repertoire, I decided to learn a functional language; at the recommendation of a friend, I selected Haskell. Thus far, it seems great. As a mathematician at heart, I love the way that the notation and language constructs resemble math (list comprehensions, tuples, function composition, etc). In this blog post, I will outline the major resources I am using to learn Haskell. To learn Haskell, I am using the ebook Learn You a Haskell for Great Good.

Factorization and Divisor Count

Mon, 14 Jul 2014 11:59:15 +0000

How many divisors are there of the number $1281942112$? It turns out that determining the answer to this problem is (at most) only as difficult as determining the prime factorization of the number. In this blog post, I will outline a solution to this (and similar) problems.

The Math

The Fundamental Theorem of Arithmetic guarantees each positive integer greater than $1$ a unique prime factorization. We write this factorization as:

$$N = p_0^{e_0}p_1^{e_1}\cdots p_n^{e_n}$$

where $p_k$ is a prime number, and $e_k$ is its corresponding exponent. This provides us with useful information regarding divisors of $N$: any divisor of $N$ must be comprised of some combination of those prime factors (and exponents). Specifically, we can define the divisor, $D$, as:

$$D = p_0^{a_0}p_1^{a_1}\cdots p_n^{a_n}$$

where the $p_k$ are the same as in the factorization of $N$ and $a_k \in {0, 1, \ldots, e_k}$. To find the total number of divisors, we multiply together the number of options we have for each exponent. That is,

$$\text{Number of Divisors}; = (e_0+1)(e_1+1)\cdots(e_n + 1)$$

Example: Consider $N = 20$. In this case, $N$ has $6$ divisors; to determine this without needing to list them all, we may note that $N = 2^2\cdot 5^1$. Using the notation described above, this means that $p_0 = 2,;p_1 = 5$ and $e_0 = 2;e_1 = 1$. Each of our divisors will be of the form $2^{a_0}\cdot 5^{a_1}$, where $a_0$ could be $0, 1,$ or $2$ and $a_1$ could be $0$ or $1$. Since we have $e_0+1 = 3$ options for $a_0$ and $e_1+1 = 2$ options for $a_1$, we have $3\cdot 2 = 6$ total divisors. In case you were wondering, the list of divisors is:

$${2^0 5^0, 2^1 5^0,2^2 5^0,2^0 5^1,2^1 5^1,2^2 5^1}$$

"Big-O" notation: An Introduction to Asymptotics of Loops

Mon, 09 Jun 2014 23:28:45 +0000

Algorithmic efficiency is imperative for success in programming competitions; your programs must be accurate and fast. To help evaluate algorithms for speed, computer scientists focus on what is called “asymptotics,” or “asymptotic analysis.” The key question answered by asymptotics is: “When your input gets really big, how many steps does your program take?” This post seeks to explain basic asymptotic analysis and its application to computing simple program runtime.

The underlying principle of asymptotic analysis is that a program’s runtime depends on the number of elementary operations it performs. The fewer elementary operations, the faster the program (and vice-versa). What do I mean by “elementary operation?” By this, I refer to any operation such that the runtime is not affected by the input size. This is more commonly referred to as a constant-time operation. Examples of such operations are assignment, basic arithmetic operations (+, -, *, /, %), accessing an array element, increment/decrement operations, function returns, and boolean expressions.

A First Example

So, a good way of gauging the runtime of a program is to count the number of elementary operations it performs. Let’s jump right in by analyzing a simple program.

1public static int test(int N) {
2 int i = 0;
3
4 while(i < N) {
5 i++;
6 }
7 return i;
8}

Obviously, this program always returns $N$, so the loop is unnecessary. However, let’s just analyze the method as-is.

Lines 2 and 7 each contribute one constant-time operation. The loop contributes two constant-time operations per iteration (one for the comparison, one for the increment), plus one extra constant-time operation for the final comparison that terminates the loop. So, the total number of operations is:

$$1 + 1 + \underbrace{\sum_{i = 0}^N 2}_{\text{loop operations}} + 1 = 3 + 2N$$

(Notice how I used sigma (summation) notation for counting a loop’s operation. This is useful, because loops and sigma notation behave in much the same way.)

Thus, it will take $3+2N$ operations to perform that method, given an input $N$. If each operation takes $2\times 10^{-9}$ (about the speed of a 2 GHz processor), it would take 5 seconds to run this program for an input of $N=10^{10}$.

Green's Theorem and the Area of Polygons

Thu, 05 Jun 2014 18:48:45 +0000

I am an avid member of the Math.StackExchange community. We have recently reached a milestone, as our request to create a site blog has been approved by the Stack Exchange administration. I volunteered to write a post which I believe should be useful to competition programmers. Using Green’s Theorem, this post derives a formula for the area of any simple polygon, dependent solely on the coordinates of the vertices. This is useful for some computational geometry problems in programming; for example, the formula can be used to compute the area of the convex hull of a set of points.

Hello World!

Fri, 23 May 2014 23:58:00 +0000

After managing a fairly successful blog for many years about competitive robotics, I am attempting to re-brand myself as I begin my studies in the field of Computer Science and Mathematics. This blog will be the place I post interesting pieces of code I either develop or find, as well as math concepts useful to competitive programmers. This blog will focus heavily on ACM-style competitions, and may occasionally contain hints or my solutions to problems from sites like USACO or UVA Online Judge.