Jekyll2020-06-07T21:12:21+00:00http://shanky.org/feed.xmlShashank TiwariShashank Tiwari's Personal Website http://www.shanky.orgShashank TiwariWhat are Indeterminate Forms?2020-06-07T00:00:00+00:002020-06-07T00:00:00+00:00http://shanky.org/2020/06/07/what-are-indeterminate-forms<script type="text/javascript" async="" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-MML-AM_CHTML">
</script>
<p>The limit of a function at a point <strong>a</strong> in its domain is the value it approaches as its argument approaches <strong>a</strong>. Intutively and informally, one could say that the limit of a function of a variable <strong>x</strong> at <strong>a</strong> is <strong>L</strong> if the function gets closer and closer to <strong>L</strong> as <strong>x</strong> gets closer to <strong>a</strong>.</p>
<script type="math/tex; mode=display">\lim_{x \to a} f(x) = L</script>
<p>For more on limits please read <a href="https://tutorial.math.lamar.edu/classes/calci/thelimit.aspx" target="_blank">Paul’s Online Notes on the subject</a>.</p>
<p>The concept of limit is fundamental to calculus and serves as the building block to define derivatives and definite integrals.</p>
<h2 id="limit-of-an-expression-involving-multiple-functions">Limit of an expression involving multiple functions</h2>
<p>The limit of an expression involving multiple functions can be obtained by taking the limit of the individual functions and then combining them to get the limit of the expression. Here is a simple example:</p>
<script type="math/tex; mode=display">\lim_{x \to a} f(x) = L1</script>
<script type="math/tex; mode=display">\lim_{x \to a} g(x) = L2</script>
<script type="math/tex; mode=display">\lim_{x \to a} (f(x)+g(x)) = L1 + L2</script>
<script type="math/tex; mode=display">\lim_{x \to a} \frac{f(x)}{g(x)} = \frac{L1}{L2}</script>
<p>In some cases though this trick leads to unresolvable expressions. Common among such unresolvable expressions are limits of functions</p>
<script type="math/tex; mode=display">\lim_{x \to a} \frac{f(x)}{g(x)}</script>
<p>leading to ratios of</p>
<script type="math/tex; mode=display">\frac{0}{0} and \frac{\infty}{\infty}.</script>
<p>This is when both <strong>L1</strong> and <strong>L2</strong> in our example above are either 0 or <script type="math/tex">\infty</script> at the same time. Limits of such expressions cannot be determined only by the limits of the functions in the expression. Such an expression is in an <strong>indeterminate form</strong>.</p>
<h2 id="examples-of-indeterminate-forms">Examples of indeterminate forms</h2>
<script type="math/tex; mode=display">\lim_{x \to 0} \frac{kx}{x}</script>
<p>where k is a real number and</p>
<script type="math/tex; mode=display">\lim_{x \to 0} \frac{sin(x)}{x}</script>
<p>leads to an expression of form <script type="math/tex">\frac{0}{0}</script>.</p>
<p>Similarily,</p>
<script type="math/tex; mode=display">\lim_{x \to \infty} \frac{kx}{x}</script>
<p>and</p>
<script type="math/tex; mode=display">\lim_{x \to \infty} \frac{log(x)}{x}</script>
<p>creates an expression of form <script type="math/tex">\frac{\infty}{\infty}</script></p>
<h2 id="the-7-indeterminate-forms">The 7 indeterminate forms</h2>
<p>There are 7 different indeterminate forms. <script type="math/tex">\frac{0}{0}</script> and <script type="math/tex">\frac{\infty}{\infty}</script> are the most common out of these 7. Other indeterminate forms are <script type="math/tex">0\times\infty</script>, <script type="math/tex">\infty - \infty</script>, <script type="math/tex">0^0</script>, <script type="math/tex">1^\infty</script>, and <script type="math/tex">\infty^0</script>.</p>
<blockquote>
<p>Note: <script type="math/tex">\frac{0}{\infty}</script> is not an indeterminate form. If <script type="math/tex">\lim_{x \to \infty} f(x) = 0</script> and <script type="math/tex">\lim_{x \to \infty} g(x) = \infty</script>, then <script type="math/tex">\lim_{x \to \infty} \frac{f(x)}{g(x)}</script> is of the form <script type="math/tex">\frac{0}{\infty}</script>. If you however rewrite this equation in the form <script type="math/tex">\lim_{x \to \infty} f(x)\times\frac{1}{g(x)}</script> it is easy to see that both the multiplicands are tending to <script type="math/tex">0</script>. <script type="math/tex">0 \times 0</script> is <script type="math/tex">0</script> and hence the limit of this expression is <script type="math/tex">0</script>.</p>
</blockquote>
<p>For forms of the type <script type="math/tex">\frac{0}{0}</script>, one could use the definition of a derivative to resolve the situation. By defintion, derivative <script type="math/tex">f'(x)</script> is</p>
<script type="math/tex; mode=display">\lim_{h \to 0} \frac{f(x+h) - f(x)}{h}</script>
<p>which is of the form <script type="math/tex">\frac{0}{0}</script>.</p>
<h2 id="lhospitals-rule">L’Hospital’s Rule</h2>
<p>L’Hospital’s rule tells us that if we have an expression of the indeterminate form <script type="math/tex">\frac{0}{0}</script> or <script type="math/tex">\frac{\infty}{\infty}</script> then we must differentiate the numerator and differentiate the denominator separately and then take the limit.</p>
<p>In this case we get:</p>
<script type="math/tex; mode=display">\lim_{x \to a} \frac{f(x)}{g(x)} = \frac{f'(x)}{g'(x)}</script>
<p>As an example,</p>
<script type="math/tex; mode=display">\lim_{x \to 0} \frac{sin(x)}{x} = \lim_{x \to 0} \frac{cos(x)}{1} = \frac{1}{1} = 1</script>
<p>L’Hospital’s rule can also be applied to indeterminate expressions of the form <script type="math/tex">0\times\infty</script>. In fact, such an expression can easily be converted to <script type="math/tex">\frac{0}{0}</script> or <script type="math/tex">\frac{\infty}{\infty}</script> form.</p>
<script type="math/tex; mode=display">\lim_{x \to a} f(x) = 0</script>
<script type="math/tex; mode=display">\lim_{x \to a} g(x) = \infty</script>
<script type="math/tex; mode=display">\lim_{x \to a} {f(x)}{g(x)}</script>
<p>is then of the form <script type="math/tex">0\times\infty</script>.</p>
<p>A simple transformation to <script type="math/tex">\lim_{x \to a} \frac{f(x)}{\frac{1}{g(x)}}</script> converts it to <script type="math/tex">\frac{0}{0}</script> form.</p>
<p>For a more detailed explanation and a few examples of the application of L’Hospital’s rule please read <a href="https://tutorial.math.lamar.edu/Classes/CalcI/LHospitalsRule.aspx" target="_blank">Paul’s Online Notes on the subject</a> or <a href="https://www.math24.net/lhopitals-rule/" target="_blank">math24.net’s tutorial on L’Hospital’s rule</a>.</p>
<h2 id="solving-indeterminate-forms">Solving indeterminate forms</h2>
<p>L’Hospital’s rule is the best method to solve indeterminate forms. In some cases an initial transformation allows an expression to be converted to <script type="math/tex">\frac{0}{0}</script> or <script type="math/tex">\frac{\infty}{\infty}</script> form after which the L’Hospital’s rule can be applied to the expression.</p>
<p>In some other cases, transformation could lead to forms where one can easily calculate the limits. Here is an example:</p>
<script type="math/tex; mode=display">\lim_{x \to \infty} (\sqrt{x^2+x+5} - x) = \lim_{x \to \infty} (\sqrt{x^2+x+5} - x) \times \frac{\sqrt{x^2+x+5} + x}{\sqrt{x^2+x+5} + x}</script>
<script type="math/tex; mode=display">= \lim_{x \to \infty} \frac{x^2+x+5 - x^2}{\sqrt{x^2+x+5} + x} = \lim_{x \to \infty} \frac{x+5}{\sqrt{x^2+x+5} + x} = \lim_{x \to \infty} \frac{\frac{x}{x}+\frac{5}{x}}{\sqrt{\frac{x^2}{x^2}+\frac{x}{x^2}+\frac{5}{x^2}} + \frac{x}{x}}</script>
<script type="math/tex; mode=display">= \lim_{x \to \infty} \frac{1+\frac{5}{x}}{\sqrt{1+\frac{1}{x}+\frac{5}{x^2}} + 1} = \frac{1}{2}</script>
<h2 id="conclusion">Conclusion</h2>
<p>There are 7 different types of indeterminate forms. In order to find the limits of expressions in indeterminate forms, transform them to a <script type="math/tex">\frac{0}{0}</script> or <script type="math/tex">\frac{\infty}{\infty}</script> form and apply the L’Hospital’s rule.</p>tshankyAn Indeterminate form is an expression involving two functions whose limit cannot be determined solely from the limits of the individual functions.Reading Files Using Golang2020-01-01T00:00:00+00:002020-01-01T00:00:00+00:00http://shanky.org/2020/01/01/reading-files-using-golang<p>From reading configuration to input data, reading files is a common operation in any programming language. Golang supports it too!</p>
<p>In this tutorial, we will learn how to use Golang to read files by reading files using Golang. In short, we will learn by doing.</p>
<p>I am assuming that you have Golang and all its required environment variables setup. If not then follow the instructions in <a href="https://golang.org/doc/install" target="_blank">the official Getting Started guide</a> to install and setup golang first. If you are on a mac, consider using <a href="https://formulae.brew.sh/formula/go" target="_blank">brew to install golang</a>.</p>
<h2 id="reading-an-entire-file-in-memory">Reading an Entire File in Memory</h2>
<p>We will start with a very small and simple plain text file. A file that contains a single line quote from Amelia Earheart. The file, <code class="language-plaintext highlighter-rouge">data.txt</code>, is only 66 bytes in size. The contents can easily fit in memory.</p>
<p>Here are the contents of <code class="language-plaintext highlighter-rouge">data.txt</code>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>‘The most effective way to do it, is to do it.’ Amelia Earhart
</code></pre></div></div>
<p>Navigate to the home directory of your go workspace. The directory that <code class="language-plaintext highlighter-rouge">$GOPATH</code> points to and then navigate to the <code class="language-plaintext highlighter-rouge">src</code> folder there. You should have 3 folders at your <code class="language-plaintext highlighter-rouge">$GOPATH</code>: <code class="language-plaintext highlighter-rouge">src</code>, <code class="language-plaintext highlighter-rouge">bin</code>, and <code class="language-plaintext highlighter-rouge">pkg</code>. Within the <code class="language-plaintext highlighter-rouge">src</code> folder, create a new directory named <code class="language-plaintext highlighter-rouge">readfile</code> and then create a file named <code class="language-plaintext highlighter-rouge">readfile.go</code> within this new directory. In <code class="language-plaintext highlighter-rouge">readfile.go</code> add the following contents:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>package main
import (
"fmt"
"io/ioutil"
)
func main() {
data, err := ioutil.ReadFile("data.txt")
if err != nil {
fmt.Println("Error: ", err)
return
}
fmt.Println("File data:", string(data))
}
</code></pre></div></div>
<p>This program when built and run is programmed to read a file named <code class="language-plaintext highlighter-rouge">data.txt</code>. When you run this program from a directory that has <code class="language-plaintext highlighter-rouge">data.txt</code> the output is as follows:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./readfile
File data: ‘The most effective way to do it, is to do it.’ Amelia Earhart
</code></pre></div></div>
<blockquote>
<p><a href="https://golang.org/pkg/io/ioutil/#example_ReadFile">ioutil.ReadFile</a> reads an entire file and outputs the contents as <code class="language-plaintext highlighter-rouge">byte</code> slices. The method signature spells this out clearly: <code class="language-plaintext highlighter-rouge">func ReadFile(filename string) ([]byte, error)</code>.</p>
</blockquote>
<p>However, running it from a different directory, one that does not have <code class="language-plaintext highlighter-rouge">data.txt</code> locally, gives an error:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./readfile
Error: open data.txt: no such file or directory
</code></pre></div></div>
<p>This can be resolved by explicitly specifying the path to the file.</p>
<h2 id="reading-a-file-at-a-given-path">Reading a File at a Given Path</h2>
<p>If <code class="language-plaintext highlighter-rouge">data.txt</code> is at <code class="language-plaintext highlighter-rouge">/Users/auser/path/to/data/file</code> then one could specify this fully qualified path in the program as follows:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>data, err := ioutil.ReadFile("/Users/auser/path/to/data/file/data.txt")
</code></pre></div></div>
<p>This simple change resolves the issue and allows you to run the program from anywhere in your system. However, if you want to run this program on any other system, you will need this data file exactly at the same path as the one specified.</p>
<p>That seems like a rigid constraint so a better alternative would be to pass the file to the program as a command line argument. Here is a version of the same file reader program with the file path passed as an argument:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>package main
import (
"flag"
"fmt"
"io/ioutil"
)
func main() {
fileptr := flag.String("path", "data.txt", "file path")
flag.Parse()
data, err := ioutil.ReadFile(*fileptr)
if err != nil {
fmt.Println("Error: ", err)
return
}
fmt.Println("File data:", string(data))
}
</code></pre></div></div>
<p>Golang’s <a href="https://golang.org/pkg/flag/">flag package</a> enables command line flag parsing. Using its methods passed in arguments can be parsed as string, integer, or boolean values. The file path is passed in and parsed as a string. The flag package has a number of convenience methods to support declaration and parsing of flags for different data types. <a href="https://golang.org/pkg/flag/#FlagSet.String">flag.String</a> defines a string flag with specified name, value, and usage string. In our example, these are “path”, “data.txt”, and “file path” respectively. <code class="language-plaintext highlighter-rouge">flag.Parse()</code> is called to parse the command line into the defined flags.</p>
<p><code class="language-plaintext highlighter-rouge">flag.String</code> returns the address of a string variable that stores the value of the flag. In short, a pointer to the flag value. That pointer is passed to <code class="language-plaintext highlighter-rouge">ioutil.Readfile</code> to read the contents of the file.</p>
<p>Using this method one could specify either the absolute or a relative path to the file being parsed. This makes your code portable across systems. Just pass in the path to the file on that system as follows:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./readfile -path /path/to/data.txt
</code></pre></div></div>
<h2 id="packing-it-in-with-the-binary">Packing it in with the Binary</h2>
<p>In some cases, it may make sense to bake the file in with the binary. In those cases you simply run your program without worrying about specifying the file path. There are <a href="https://tech.townsourced.com/post/embedding-static-files-in-go/">multiple ways of embedding static files in go</a>. Baking in makes sense when the files are static and seldom change. For our example, lets continue with our small 66 bytes file. In real projects though, such files are likely to be static assets like images, css and js files, message bundles for internationalization, or internal configuration files that change with newer releases only.</p>
<p>There are a number of packages available to embed static files. The simplest of these is probably <a href="https://github.com/rakyll/statik">statik</a>. Install <code class="language-plaintext highlighter-rouge">statik</code> and use it to convert the data file to a go file.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>go get github.com/rakyll/statik
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">statik</code> accepts a directory as input to translate to go format. Create a directory in your project src folder (<code class="language-plaintext highlighter-rouge">src/readfile</code>) and name it <code class="language-plaintext highlighter-rouge">data</code>. Move data.txt to this new directory. Then convert the data file to go format.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>cd src/readfile
statik -src=./data
</code></pre></div></div>
<p>A directory with go translated files for static assets is generated in a directory named <code class="language-plaintext highlighter-rouge">statik</code>. It contains a file within it called <code class="language-plaintext highlighter-rouge">statik.go</code>. The generated contents are in a package named <code class="language-plaintext highlighter-rouge">statik</code>.</p>
<p>Read the embedded data file as follows:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>package main
import (
"fmt"
"io/ioutil"
"log"
_ "readembeddedfile/statik"
"github.com/rakyll/statik/fs"
)
func main() {
statikFS, err := fs.New()
if err != nil {
log.Fatal(err)
}
data, err := statikFS.Open("/data.txt")
if err != nil {
fmt.Println("Error: ", err)
return
}
defer data.Close()
contents, err := ioutil.ReadAll(data)
if err != nil {
log.Fatal(err)
}
fmt.Println("File data:", string(contents))
}
</code></pre></div></div>
<p>This code is similar to our original file reading code, except that it involves pre-processing to convert text file to go file and then reading of that data using a <code class="language-plaintext highlighter-rouge">statik</code> package specific filesystem abstraction, named <code class="language-plaintext highlighter-rouge">statikFS</code>.</p>
<p>Embedding static content also makes your go binary much larger. The orginal readfile binary is a little over 2MB in size whereas the one with the embedded data is over 7MB.</p>
<h3 id="reading-larger-files">Reading Larger Files</h3>
<p>For small files, there is no problem reading it all up in memory and then processing it. For larger files you could run out of memory. Bigger files should be read either as byte chunks or line by line. In either case, you need to buffer the content, read a small part, and continue to read addiitonal small parts until you have exhausted all the content in the file. The golang <code class="language-plaintext highlighter-rouge">bufio</code> package provides functionality to read both as byte chunks and line by line.</p>
<p>For text files, reading line by line is sometimes the most apprproiate way to consume the content. Our data file in the example has just one line so lets add another line to it to demonstrate line by line reading. The new data file has 2 lines like so:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>‘The most effective way to do it, is to do it.’ Amelia Earhart
'Talk is cheap. Show me the code.' Linus Torvalds
</code></pre></div></div>
<p>To read line by line, use the following code:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>package main
import (
"bufio"
"flag"
"fmt"
"log"
"os"
)
func main() {
fileptr := flag.String("path", "data.txt", "file path")
flag.Parse()
f, err := os.Open(*fileptr)
if err != nil {
log.Fatal(err)
}
defer func() {
if err = f.Close(); err != nil {
log.Fatal(err)
}
}()
s := bufio.NewScanner(f)
for s.Scan() {
fmt.Println(s.Text())
}
err = s.Err()
if err != nil {
log.Fatal(err)
}
}
</code></pre></div></div>
<p>You will notice that <code class="language-plaintext highlighter-rouge">os.Open</code> is used in place of <code class="language-plaintext highlighter-rouge">ioutil.ReadFile</code>. A <code class="language-plaintext highlighter-rouge">NewScanner</code> in the <code class="language-plaintext highlighter-rouge">bufio</code> package is instantiated and used to read the contents line by line. The file is closed once it reaches the end.</p>
<p>Reading by chunks is similar:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>package main
import (
"bufio"
"flag"
"fmt"
"log"
"os"
)
func main() {
fileptr := flag.String("path", "data.txt", "file path")
flag.Parse()
f, err := os.Open(*fileptr)
if err != nil {
log.Fatal(err)
}
defer func() {
if err = f.Close(); err != nil {
log.Fatal(err)
}
}()
r := bufio.NewReader(f)
b := make([]byte, 8)
for {
n, err := r.Read(b)
if err != nil {
fmt.Println("Error:", err)
break
}
fmt.Println(string(b[0:n]))
}
}
</code></pre></div></div>
<p>The difference is to use a <code class="language-plaintext highlighter-rouge">bufio.NewReader</code> in place of a <code class="language-plaintext highlighter-rouge">bufio.NewScanner</code>. Byte chunk size is specified and the content is read in those chunk slices. In the example above the size is set as 8 bytes.</p>
<h3 id="what-next">What Next?</h3>
<p>By now you can read small files and large ones. Using embedding, you know you are able to distribute static content with your binary. You could use the concepts of chunk based or line by line reading to read humungous files.</p>
<p>You could also use golang to read very large files by splitting and reading parts of it in parallel. That may be a good topic for another post.</p>
<p>Hope you enjoyed reading. Ask questions, if you have any, and leave comments to share your thoughts.</p>tshankyFrom reading configuration to input data, reading files is a common operation in any programming language. Learn to read files effectively using golang.Understanding the Birthday Paradox2018-11-08T00:00:00+00:002018-11-08T00:00:00+00:00http://shanky.org/2018/11/08/understanding-the-birthday-paradox<script type="text/javascript" async="" src="https://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-MML-AM_CHTML">
</script>
<p>By definition, a paradox is a seemingly absurd statement or proposition that when investigated or explained may prove to be well-founded and true.</p>
<p>It’s hard to believe that there is more than <strong>50% chance</strong> that at least 2 people in a group of randomly chosen <strong>23</strong> people have the same birthday. Further, if the group had 75 people then its almost certain that 2 of them were born on the same day of the year. The number 23 appears too small for such a high probability that at least 2 people have the same birthday, and hence the paradox. In this post we will reason through the birthday problem and learn and understand the mathematics behind the chance of sharing birthdays.</p>
<h2 id="what-are-the-chances-at-least-2-people-out-of-23-people-in-a-room-have-the-same-birthday">What are the Chances at least 2 people out of 23 People in a Room Have the Same Birthday?</h2>
<p>The number of different pairs one can form with 23 people is the number of ways you can select 2 people from a total of 23. The general formula for choosing <code class="language-plaintext highlighter-rouge">k</code> things from a set of <code class="language-plaintext highlighter-rouge">n</code> things is:</p>
<script type="math/tex; mode=display">\binom{n}{k}</script>
<p>Using this formula you can calculate that there are 253 different pairs of people among a set of 23 people.</p>
<script type="math/tex; mode=display">\binom{23}{2} = \frac{23!}{2!(23-2)!} = \frac{23\times22}{2} = 253</script>
<blockquote>
<p>If you are confused about the combination formula or do not understand factorial, you will benefit from reviewing the fundamentals of combination. Watch the <a href="https://www.khanacademy.org/math/precalculus/prob-comb#combinations" title="Combinations" target="_blank">Khan Academy video on the subject</a> or read the <a href="https://en.wikipedia.org/wiki/Combination" title="Combination" target="_blank">Wikipedia page on combination</a>.</p>
</blockquote>
<p>For a given pair, the probability that those two people have the same birthday is <code class="language-plaintext highlighter-rouge">1/365</code>. Therefore, the probability that the two have different birthdays is <code class="language-plaintext highlighter-rouge">364/365</code>, <code class="language-plaintext highlighter-rouge">1 - 1/365</code>. There are 365 days in the year. 1 out of these 365 days is when the two have the same birthday and the other 364 days is when their birthdays don’t match.</p>
<blockquote>
<p>We have ignored the case of a leap year, when there are 366 days in a year. The math here wouldn’t quite hold for people born on February 29th.</p>
</blockquote>
<p>The probability that none of the 253 pairs share birthdays is:</p>
<script type="math/tex; mode=display">\left(\frac{364}{365}\right)^{253} = 0.499522846</script>
<p>This implies that the probability that at least one of these pairs share a birthday is <code class="language-plaintext highlighter-rouge">1 - 0.499522846</code>, which is <code class="language-plaintext highlighter-rouge">0.500477154</code>.</p>
<p>In other words, the chance that one or more of these pairs, out of a total of 253 pairs, share a birthday, is greater than 50%. 50.0477% to be precise.</p>
<h2 id="how-does-this-work-for-n-people-in-a-room">How Does this Work for n People in a Room?</h2>
<p>What if there weren’t 23 people in the room? What if there were 45, or 15, or 80 people in the room? To calculate the probabilities for all these cases and many more, let’s consider the generic variable <code class="language-plaintext highlighter-rouge">n</code>.</p>
<p>You can select pairs of people among n people as follows:</p>
<script type="math/tex; mode=display">\binom{n}{2} = \frac{n!}{2!(n-2)!} = \frac{n\times(n-1)}{2}</script>
<p>Following the same reasoning as for the case of 23 people, the probability that all pairs among these n people have different birthdays is:</p>
<script type="math/tex; mode=display">\left(\frac{364}{365}\right)^{\frac{n\times(n-1)}{2}}</script>
<p>The general probability function to calculate the chance that at least one of these pairs have the same birthday is:</p>
<script type="math/tex; mode=display">p(n) = 1 - \left(\frac{364}{365}\right)^{\frac{n\times(n-1)}{2}}</script>
<p>p(45) is 0.9338645976, a 93.3864% chance of a match, p(15) is 0.2502879086, around 25%, and p(80) is 0.9998282405 or 99.9828%, which implies almost a certain match.</p>
<h2 id="are-birthdays-of-any-two-people-among-a-set-of-n-people-independent">Are Birthdays of any two People Among a set of n People Independent?</h2>
<p>In our formula for calculating probability of at least one pair with a matching birthday, among all possible pairs from a total of <code class="language-plaintext highlighter-rouge">n</code> people, we stated that the probability of no match is:</p>
<script type="math/tex; mode=display">\left(\frac{364}{365}\right)^{\frac{n\times(n-1)}{2}}</script>
<p>This formula is built on 2 assumptions:</p>
<ol>
<li>The distribution of birthdays is uniform across the sample space. That is, the probability that someone’s birthday is on any of the 365 possible days is equally likely. Empirical studies have shown that this statement is not true. Months between July and October typically have higher births. <a href="https://www.panix.com/~murphy/bday.html" title="On uniformity of distribution of birthdays in a calendar year" target="_blank">Here is</a> one interesting write-up summarizing the reasonableness of the assumption of uniformity of distribution of birthdays in a calendar year.</li>
<li>For all possible pairs among the n people, the chance that any one pair has matching birthday is independent of all the other pairs. However, reason through a bit and you will soon realize that its not the case. Let’s assume Alice and Bob have the same birthday and Bob and Charlie have the same birthday. Applying transitivity, its clear that the match for these 2 pairs also implies that Alice and Charlie have the same birthday. Therefore, the chance of match for all possible pairs is not independent of each other. <a href="https://en.wikipedia.org/wiki/Independence_(probability_theory)" title="Independence" target="_blank">In probability theory, independence of events or occurrences</a> allows us to calculate joint probabilities by simply multiplying their probabilities, which is what we do in our formula.</li>
</ol>
<p>Let’s rework the case of 23 people in a room. The more accurate joint probability of each of the 23 people having a different birthday is:</p>
<script type="math/tex; mode=display">1\times\left(1 - \frac{1}{365}\right)\times\left(1 - \frac{2}{365}\right)\times\left(1 - \frac{3}{365}\right)...\times\left(1 - \frac{22}{365}\right)</script>
<p>Why? Start with any one person among those 23. The chance that the person has a unique birthday is 100%. For the second person, the chance of having a different birthday effectively means having a birthday on any of the other days other than the birthday of the first person, which is <script type="math/tex">1 - \left(\frac{1}{365}\right)</script>. Follow the same logic for the next set of people and you will find that the formula we just wrote is obvious.</p>
<p>While, this formula is more accurate, it is a bit complicated as compared to our formula under the assumption of independence of events. Let’s use some mathematical tools that will help us approximate this formula.</p>
<p>According to <a href="https://en.wikipedia.org/wiki/Taylor_series" title="Taylor Series" target="_blank">Taylor Series</a> an exponential function can be written as an infinite sum of terms as follows:</p>
<script type="math/tex; mode=display">e^x = 1 + x + \frac{x^2}{2!} + ....</script>
<p>When x is very small, the exponential function can be approximated to the first couple of terms of this series.</p>
<script type="math/tex; mode=display">e^x \approx 1 + x</script>
<blockquote>
<p>Will explain Taylor Series and its applications in a subsequent post.</p>
</blockquote>
<p>Applying Taylor Series approximation for the exponential function</p>
<script type="math/tex; mode=display">e^\frac{-1}{365} \approx \left(1 - \frac{1}{365}\right)</script>
<p>Our complex joint probability function then takes a shape like so:</p>
<script type="math/tex; mode=display">1\times(e^\frac{-1}{365})\times(e^\frac{-2}{365})\times(e^\frac{-3}{365})...\times(e^\frac{-22}{365})</script>
<p><script type="math/tex">e^0 = 1</script> so the probability that no pairs among 23 people have matching birthdays is approximated to</p>
<script type="math/tex; mode=display">e^\frac{(-1-2-3...-22)}{365} = e^\frac{-(22\times23)}{2\times365} = 0.499998</script>
<p>Fairly close to the value, 0.499522846, we calculated assuming independence of events.</p>
<blockquote>
<p>The sequence of first n integers forms an <a href="https://en.wikipedia.org/wiki/Arithmetic_progression" title="Arithmetic progression" target="_blank">Arithmetic progression</a>. The sum of such a progression of first n integers is <script type="math/tex">\frac{n\times(n+1)}{2}</script>.</p>
</blockquote>
<p>Using this approach, the general approximated formula for at least one pair with same birthday among <code class="language-plaintext highlighter-rouge">n</code> people is:</p>
<script type="math/tex; mode=display">p(n) \approx 1 - e^\frac{-(n\times(n+1))}{2\times365}</script>
<h2 id="what-about-martians">What About Martians?</h2>
<p>The average length of a solar day, also known as sol, on Mars is 24 hours, 39 minutes, and 35.244147 seconds. <a href="https://en.wikipedia.org/wiki/Timekeeping_on_Mars" title="Timekeeping on Mars" target="_blank">A year on Mars is about 668.5991 sols</a>, which is roughly around 686.98 Earth solar days. Let’s approximate it to 668 Martian sols for our calculation.</p>
<p>Therefore, the probability that at least 2 Martians among a room full of <code class="language-plaintext highlighter-rouge">n</code> Martians share a birthday is:</p>
<script type="math/tex; mode=display">p(n) \approx 1 - e^\frac{-(n\times(n+1))}{2\times668}</script>
<p>This means you need 31 Martians in a room so that there is greater than 50% chance that at least 2 of them share a birthday.</p>
<h2 id="the-birthday-problem-formula">The Birthday Problem Formula</h2>
<p>The general formula we have so far</p>
<script type="math/tex; mode=display">p(n) \approx 1 - e^\frac{-(n\times(n+1))}{2\times365}</script>
<p>could be approximated further by dropping the lower powers of n in the exponential. The formula then takes the following form:</p>
<script type="math/tex; mode=display">p(n) \approx 1 - e^\frac{-(n^2)}{2\times365}</script>
<p>365 is the total number of possibilities for number of days on Earth. We noticed that it is 668 for Mars. To generalize the birthday problem to finding at least 1 match among pairs chosen from <code class="language-plaintext highlighter-rouge">n</code> things out of a total number of <code class="language-plaintext highlighter-rouge">U</code> things in a sample space or universe, we get the following:</p>
<script type="math/tex; mode=display">p(n) \approx 1 - e^\frac{-(n^2)}{2\times(U)}</script>
<p>If the desired probability was 50%, i.e. p(n) = 0.5, then we could approximate to:</p>
<script type="math/tex; mode=display">0.5 \approx 1 - e^\frac{-(n^2)}{2\times(U)}</script>
<script type="math/tex; mode=display">-0.5 \approx e^\frac{-(n^2)}{2\times(U)}</script>
<p>taking the natural log on both sides</p>
<script type="math/tex; mode=display">-2\log_{e}(0.5)\times(U) \approx n^2</script>
<script type="math/tex; mode=display">n \approx 1.177\sqrt{U}</script>
<h2 id="what-are-the-chances-of-somebody-else-in-the-same-room-having-the-same-birthday-as-you">What are the Chances of Somebody Else in the Same Room Having the Same Birthday as you?</h2>
<p>This question can be a bit deceiving. Many confuse this question with the more common one of finding at least 1 match among all possible pairs in a room full of <code class="language-plaintext highlighter-rouge">n</code> people!</p>
<p>When you are finding possible matches for your birthday among <code class="language-plaintext highlighter-rouge">n</code> people in a room, you are looking at only <code class="language-plaintext highlighter-rouge">n-1</code> pairs and not <script type="math/tex">\binom{n}{2}</script> pairs. This means, whereas there are 253 possible pairs among 23 people in a room, there are only 22 pairs possible where you are a member of that pair.</p>
<p>This changes the mathematics drastically.</p>
<p>Whereas you need only 23 people to have a greater than 50% chance that at least 1 pair chosen from these 23 people share a birthday, you need 253 people other than yourself in a room to have a 50% chance that someone has the same birthday as you in the room. Extending the calculation further, you need 2775 people in a room to have greater than 99.9506% chance of a match. This same chance is achieved with only 75 people, when the problem seeks to achieve a possible match, not necessarily with you but with anyone in the room.</p>
<p>You may have noticed that the power of factorial and combination are at play</p>
<p><script type="math/tex">\binom{23}{2} = 253</script>
<code class="language-plaintext highlighter-rouge">and</code>
<script type="math/tex">\binom{75}{2} = 2775</script></p>
<h2 id="other-fun-problems-based-on-similar-ideas">Other Fun Problems Based on Similar Ideas</h2>
<p>Here are a couple of fun problems to think about:</p>
<ol>
<li>What is the probability of someone else in a room full of n people having the same birthday as me and no other person sharing a birthday?</li>
<li>What is the chance of finding at least one pair in a group of n people with birthdays within k calendar days of each other’s?</li>
</ol>
<p>If you reason it through, based on what you have just read, then you will find the answer. Will leave this as an exercise for you to enjoy.</p>
<h2 id="conclusion">Conclusion</h2>
<p>Paradoxes are counter intuitive. The birthday problem has been written about a lot and often serves as an interesting example in many probability and statistics classes. I hope you enjoyed reading this post and learnt the fundamentals behind this problem. Chime in with your questions, thoughts, or comments.</p>tshankyThe Birthday Paradox or Birthday Problem highlights a mathematical and probabilistic outcome that is intuitively hard to believe.Getting Started with NGINX2018-11-01T00:00:00+00:002018-11-01T00:00:00+00:00http://shanky.org/2018/11/01/getting-started-with-nginx<p>Nginx, often written as NGINX or nginx, is a very versatile web server that can be used as a reverse proxy, load balancer, mail proxy, and HTTP cache. Its very easy to setup and requires minimal knowledge to accomplish basic things like serving web pages.</p>
<h2 id="installing-nginx-on-macos-mojave">Installing NGINX on macOS Mojave</h2>
<p>The easiest way to install nginx on macOS Mojave is by using <a href="https://brew.sh/" title="Homebrew" target="_blank">Homebrew</a>. If you have brew installed, update brew, and install NGINX as follows:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>brew update
brew install nginx
</code></pre></div></div>
<p>Once installed, nginx can be started by simply invoke the <code class="language-plaintext highlighter-rouge">nginx</code> command.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo nginx
</code></pre></div></div>
<p>That’s it! Open <a href="http://localhost:8080" target="_blank">http://localhost:8080</a> in your favorite browser and you should see the Nginx welcome page.</p>
<h2 id="configuring-to-serve-static-content">Configuring to Serve Static Content</h2>
<p>As a pre-requisite get some static content ready to serve. Create a couple of directories, one for <em>html</em> pages and one for <em>images</em>.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sudo mkdir -p /data/www
sudo mkdir -p /data/images
</code></pre></div></div>
<blockquote>
<p>You don’t have to create the directories at <code class="language-plaintext highlighter-rouge">/data</code>. They could be anywhere, in a place of your choice, on the file system. Just make sure to update the <code class="language-plaintext highlighter-rouge">location</code> parameter in the configuration file accordingly.</p>
</blockquote>
<p>Then, put a of piece of content in each of these directories. For example, create an <em>html</em> file, named <code class="language-plaintext highlighter-rouge">index.html</code>, and add some text content to that file and put it in the <code class="language-plaintext highlighter-rouge">/data/www</code> directory. Similarly, get an image, say <code class="language-plaintext highlighter-rouge">an_image.png</code>, and put it in the <code class="language-plaintext highlighter-rouge">/data/images</code> directory.</p>
<p>Once you have your static content ready, its time to update the configuration to instruct nginx to serve content from these directories.</p>
<p>If you installed nginx using <a href="https://brew.sh/" title="Homebrew" target="_blank">Homebrew</a>, your config file, <code class="language-plaintext highlighter-rouge">nginx.conf</code>, is at <code class="language-plaintext highlighter-rouge">/usr/local/etc/nginx/</code>. Open the config file and modify the <code class="language-plaintext highlighter-rouge">http</code> directive so that it looks like so:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>http {
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
server {
server_name localhost;
location / {
root /data/www;
}
location /images/ {
root /data;
}
}
}
</code></pre></div></div>
<p>Start the server using <code class="language-plaintext highlighter-rouge">sudo nginx</code> or reload the configuration for a running server using <code class="language-plaintext highlighter-rouge">sudo nginx -s reload</code>. Your static content is now ready to be served by nginx.</p>
<p>Open your browser and simply point it to <a href="http://localhost" title="index at localhost" target="_blank">http://localhost</a> to view the contents of <code class="language-plaintext highlighter-rouge">index.html</code>. Pointing to <a href="http://localhost/images/an_image.png" title="images at localhost" target="_blank">http://localhost/images/an_image.png</a> will display your image.</p>
<h2 id="understanding-nginx-configuration">Understanding nginx Configuration</h2>
<p>Nginx consists of a number of powerful and feature rich modules. These modules are controlled by the directives in the configuration file. Understanding the structure, syntax, and semantics of the configuration file is key to leveraging nginx to your advantage.</p>
<p>Nginx configuration has a notion of hierarchical directives and instructions. Each hierarchical level forms a context. The root context in nginx terminology is the <em>main</em> context. This context includes directives for essential elements like <em>events</em> and protocols, like <em>http</em>. It also includes configuration for <em>error_log</em>, <em>worker</em> processes, <em>ssl_engine</em>, and for debugging. Some of these could be in inner nested contexts as well.</p>
<p>The <em>main</em> context can, and most likely will, have inner contexts. <em>events</em> and <em>http</em> are typical inner contexts in web server configurations.</p>
<p>Configuration directives are either <strong>simple</strong> directives or <strong>block</strong> directives. <strong>simple</strong> directives are key-value pairs written in a single line, separated by spaces, and terminated by a semi-colon (;). For such configuration, the <em>name</em> is the key and the configuration <em>parameter</em> is the value. An example of such a configuration is:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>worker_connections 1024;
</code></pre></div></div>
<p>A <strong>block</strong> directive is also a key-value pair but in this case, the value is enclosed within a pair of braces ({}). Also, a semi-colon isn’t used to terminate such a configuration directive. The enclosed set of braces can contain additional nested directives, which could be <strong>simple</strong> or <strong>block</strong> directives. The configuration we used for serving static content in the last section shows the <code class="language-plaintext highlighter-rouge">http</code> <strong>block</strong> configuration directive that has both <strong>simple</strong> and <strong>block</strong> directives within it. <code class="language-plaintext highlighter-rouge">include mime.types;</code> is a <strong>simple</strong> nested directive and the <code class="language-plaintext highlighter-rouge">server</code> directive is a <strong>block</strong> directive.</p>
<p>The <code class="language-plaintext highlighter-rouge">location</code> directive plays a special role for routing and directing requests. There can be, and often are, multiple <code class="language-plaintext highlighter-rouge">location</code> directives for a given <code class="language-plaintext highlighter-rouge">server</code> directive. The reason for having multiple <code class="language-plaintext highlighter-rouge">location</code> directives is to route requests basis URI values and patterns. This indirectly facilitates handling requests by resource type, content type, and request parameters.</p>
<p>Request URIs are matched to the longest and most specific pattern first and then matched to the next closest pattern and so on, until the only match is the shortest and catch all prefix, the forward slash(/). Regular expressions could be used in a <code class="language-plaintext highlighter-rouge">location</code> prefix allowing for definition of sophisticated patterns.</p>
<p>You will benefit from familiarizing yourself with the various configuration directives for the <a href="https://nginx.org/en/docs/ngx_core_module.html" title="Nginx Core Module Configuration" target="_blank">core module</a> and the <a href="http://nginx.org/en/docs/http/ngx_http_core_module.html" title="Nginx HTTP Core Module Configuration" target="_blank">http module</a>.</p>
<h2 id="proxying-requests">Proxying Requests</h2>
<p>You are probably familiar with a forward proxy, sometimes simply known as a proxy or a proxy server, which is an intermediary for client requests seeking resources from other servers. Forward proxies could be in the same internal network as the clients.</p>
<p>A reverse proxy is a server side counterpart to a forward proxy. A reverse proxy acts as an intermediary for one or more servers. It accepts client requests and retrieves responses from appropriate upstream servers. A reverse proxy could provide or assist in additional features like buffering, caching, and load balancing. Additionally, it could act as an application firewall.</p>
<p>Setting up nginx as a reverse proxy is easy. For a proxy where communication is exclusively over HTTP, the <code class="language-plaintext highlighter-rouge">proxy_pass</code> directive can do most of the heavy lifting. Here is an example configuration where a proxy server setup sends all requests to an upstream server supporting HTTP.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
send_timeout 1800;
sendfile on;
keepalive_timeout 6500;
server {
listen 80;
server_name localhost;
location / {
proxy_pass http://localhost:3000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Client-Verify SUCCESS;
proxy_set_header X-Client-DN $ssl_client_s_dn;
proxy_set_header X-SSL-Subject $ssl_client_s_dn;
proxy_set_header X-SSL-Issuer $ssl_client_i_dn;
proxy_read_timeout 1800;
proxy_connect_timeout 1800;
}
}
}
</code></pre></div></div>
<p>This could satisfy a case where nginx communicates via HTTP on port 80 and channels all communication to an internal web server, possibly running a Node.js or a Rails application on port 3000. In this case both the nginx server and the upstream server are running in the same environment. This isn’t a requirement. Nginx can proxy requests over to remote servers as well.</p>
<p>We will explore the powerful features around buffering, caching, and load balancing in a subsequent post. For now, you should be convinced that simple reverse proxy setup is quick and easy.</p>
<h2 id="looking-through-the-logs">Looking through the Logs</h2>
<p>By default, your nginx log files on your <a href="https://brew.sh/" title="Homebrew" target="_blank">Homebrew</a> installed nginx are at <code class="language-plaintext highlighter-rouge">/usr/local/var/log/nginx</code>. There are 2 log files in this directory: <code class="language-plaintext highlighter-rouge">access.log</code> and <code class="language-plaintext highlighter-rouge">error.log</code>. The names suggest what they contain. Information about the remote user, remote address, and request time along with the request URI, referrer, and user agent are logged in <code class="language-plaintext highlighter-rouge">access.log</code>. All errors are logged in <code class="language-plaintext highlighter-rouge">error.log</code>. Fatal emergencies leading to server crash are logged as <code class="language-plaintext highlighter-rouge">emerg</code> in the error log.</p>
<blockquote>
<p>You can change the path, logging level, and format of the log files by specifying it in <code class="language-plaintext highlighter-rouge">nginx.conf</code>. For example, you could log everything that is a warning and more severe in error.log at <code class="language-plaintext highlighter-rouge">logs/error.log</code> using a configuration as follows: <code class="language-plaintext highlighter-rouge">error_log logs/error.log warn;</code></p>
</blockquote>
<h2 id="conclusion">Conclusion</h2>
<p>You have learnt the basics of nginx in this post. We have barely scratched the surface but hopefully you are beginning to believe that nginx is both very easy to use and very powerful in terms of features.</p>tshankyGetting started with NGINXAtom editor for Python2017-01-11T00:00:00+00:002017-01-11T00:00:00+00:00http://shanky.org/2017/01/11/atom-editor-for-python<p>Github’s <a href="https://atom.io/">Atom</a> is a beautiful and customizable editor that can be leveraged gainfully for Python programming. Built on <a href="http://electron.atom.io/">Electron</a>, a technology for building cross platform desktop apps
with JavaScript, HTML, and CSS, Atom is, as Github says, “hackable”. Its support for plugins and add-ons via packages allows one to add customizations and newer capabilities beyond what comes out of the box.</p>
<h2 id="atom-packages">Atom Packages</h2>
<p>The package system is at the heart of Atom’s design. Atom itself has a very lightweight core and most of its own default features, like the “Settings View”, are implemented via packages. The out of box comes with over 90 packages installed.</p>
<p>Newer packages can be searched and installed via the “Install” tab in the “Settings View”. Look at Figure 1, which shows the “Install” tab.</p>
<figure>
<img src="/assets/images/install_tab_in_the_settings_view.png" />
<figcaption>Figure 1: Atom Settings View Install Tab</figcaption>
</figure>
<blockquote>
<p>These searchable Atom packages are published to the official Atom registry at <a href="https://atom.io/packages">https://atom.io/packages</a>.</p>
</blockquote>
<h3 id="installing-using-the-command-line">Installing using the command line</h3>
<p>Atom comes with a command line utility named <em>apm</em>, short for atom package manager. Latest version of a package can be installed easily using the following command:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>apm install <package_name>
</code></pre></div></div>
<p>This assumes you know the exact name of the package you want to install. If you are not sure of the exact name of the package then you can search for packages like so:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>apm search <possible_package_name>
</code></pre></div></div>
<p>While installing the latest version of a package is desirable, you may need to install a specific older version. Its easy to install a specific version as follows:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>apm install <package_name>@<package_version>
</code></pre></div></div>
<p>To view more information about a specific package, use:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>apm view <package_name>
</code></pre></div></div>
<h2 id="atom-packages-for-python">Atom Packages for Python</h2>
<p>PEP8 (short for Python Enhancement Proposal 8) is the style guide for Python code. Pythonic code is optimized for readability. Clean, explicit, and simple code structure, design, and layout is preferred over complexity. PEP257 specifies docstring conventions. Docstrings are string literals that appear as the first statement in a Python function, module, class, or method definition. It serves as code comments and is also accessible at runtime via the “<strong>doc</strong>” attribute. Atom’s support for code linters can help you proactively check your code for errors and adherence to PEP8 and PEP257.</p>
<p>Linter is an Atom package that provides a unified and consistent api for all atom linter plugins. Start by installing this package:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>apm install linter
</code></pre></div></div>
<p>Next, piggyback on the Python packages to do the heavy lifting. Install flake8, the wrapper around PyFlakes, pycodestyle, and Mc Cabe script. <em>pip</em> is the most popular and widely used package manager for Python and using it is very easy. To install flake8 and flake8-pep257, use pip as follows:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pip install flake8
pip install flake8-pep257
</code></pre></div></div>
<p>Now that flake8 is available on your system, install linter-flake8. The linter-flake8 package is a flake8 provider for linter. It makes it possible for Atom linter to leverage flake8 for checking Python code for style adherence. Install linter-flake8 like so:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>apm install linter-flake8
</code></pre></div></div>
<h2 id="automating-style-corrections">Automating Style Corrections</h2>
<p>Linter with flake8’s help surfaces style related errors and warnings but doesn’t correct it for you. The python package autopep8 on the other hand autocorrects your errors. Install autopep8 via the command line as follows:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pip install autopep8
</code></pre></div></div>
<p>Formatting issues identified by PEP8 checkers are listed at <a href="https://pep8.readthedocs.io/en/latest/intro.html#error-codes">PEP8 Error Codes</a>. autopep8 is capable of fixing most of these. autopep8 can be used via the command line to fix errors like so:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>autopep8 --in-place --aggressive --aggressive <filename>
</code></pre></div></div>
<p>Within an editor or a development environment its very convenient if the auto corrections are transparent and don’t involve invoking utilities via the command line. A good point for auto correction if often when a source code file is saved after it is edited. Atom’s atom-beautify package allows integration of autopep8 as the beautifier for Python code. It also offers the option to “Beautify on Save”. Install atom-beautify as follows:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>apm install atom-beautify
</code></pre></div></div>
<p>Look at Figure 2, which illustrates the configuration options for Python within atom-beautify settings.</p>
<figure>
<img src="/assets/images/atom_beautify_configuration_options_for_pyton.png" />
<figcaption>Figure 2: atom-beautify configuration options for Python</figcaption>
</figure>tshankyUsing the Atom editor for PythonOffside2016-05-06T00:00:00+00:002016-05-06T00:00:00+00:00http://shanky.org/2016/05/06/offside<p>Law 11, commonly known as the offside rule, is a rule that occupies only 1 page of the official <a href="http://www.fifa.com/development/education-and-technical/referees/laws-of-the-game.html" target="_blank">IFAB Laws of the Game</a>. Although simple and brief in appearance, its application during a match can be quite challenging. In this post, I will explain the various nuances of this law so you are better equipped to understand and apply the law.</p>
<p>Start by watching <a href="https://youtu.be/F1ugG8ZSwQM" target="_blank">this wonderful video</a> on the subject from <a href="http://www.ussoccer.com/" target="_blank">US Soccer</a>. The law can be divided into two parts, namely:</p>
<ul>
<li>Offside position</li>
<li>Offside offence</li>
</ul>
<h2 id="offside-position">Offside Position</h2>
<p>To quote <a href="http://www.fifa.com/development/education-and-technical/referees/laws-of-the-game.html" target="_blank">IFAB Laws of the Game</a>:</p>
<blockquote>
<p>Offside position</p>
</blockquote>
<p><em>It is not an offence in itself to be in an offside position.</em></p>
<p>A player is in an offside position if:</p>
<ul>
<li><em>he is nearer to his opponents’ goal line than both the ball and the second-last opponent</em></li>
</ul>
<p>A player is not in an offside position if:</p>
<ul>
<li><em>he is in his own half of the field of play or</em></li>
<li><em>he is level with the second-last opponent or</em></li>
<li><em>he is level with the last two opponents</em></li>
</ul>
<p>In the section titled: <em>Interpretation of the Laws of the Game and Guidelines for Referees</em>, a few terms in the context of the Offside law are defined.</p>
<blockquote>
<p>According to that guideline “nearer to his opponents’ goal line” means that any part of a player’s head, body or feet is nearer to his opponents’ goal line than both the ball and the second-last opponent. The arms are not included in this definition.</p>
</blockquote>
<figure>
<a href="http://i.dailymail.co.uk/i/pix/2016/04/06/19/1459969009389_lc_galleryImage_vlcsnap_00013_jpg.JPG"><img src="http://i.dailymail.co.uk/i/pix/2016/04/06/19/1459969009389_lc_galleryImage_vlcsnap_00013_jpg.JPG" /></a>
<figcaption><a href="http://i.dailymail.co.uk/i/pix/2016/04/06/19/1459969009389_lc_galleryImage_vlcsnap_00013_jpg.JPG" title="Ronaldo’s head is in an offside position">Figure 1: Ronaldo’s head is in an offside position. Since he could have scored with his head, Ronaldo is in an offside position.</a>.</figcaption>
</figure>
<p>Source: <a href="http://www.dailymail.co.uk/sport/football/article-3525441/Wolfsburg-vs-Real-Madrid-UEFA-Champions-League-LIVE-scores-UCL-Real-Madrid-vs-Wolfsburg-Cristiano-Ronaldo-aim-build-leg-quarter-final-football-lead.html" target="_blank">Wolfsburg 2-0 Real Madrid Champions League RESULT: Ricardo Rodriguez and Maximilian Arnold make La Liga giants pay for sloppy defending</a></p>
<p>Multiple players could simultaneously be in an offside position as shown in Figure 2.</p>
<figure>
<a href="http://i.imgur.com/qj3yY3d.png"><img src="http://i.imgur.com/qj3yY3d.png" /></a>
<figcaption><a href="http://i.imgur.com/qj3yY3d.png" title="5 FC Dallas Players are in an offside position">Figure 2: 5 FC Dallas Players are in an offside position.</a>.</figcaption>
</figure>
<p>Source: <a href="http://i.imgur.com/qj3yY3d.png" target="_blank">http://i.imgur.com/qj3yY3d.png</a>. With so many players being in an offside position the referees ended up making a mistake and it resulted in Blas Perez scoring a goal, while being in an offside position. A discussion on this topic is on <em>Reddit</em>: <a href="https://www.reddit.com/r/MLS/comments/2z2y5l/blas_perez_scores_his_second_goal_of_the_match">Blas Perez scores his second goal of the match with FIVE FC Dallas players, including Perez, in an offside position</a></p>
<h2 id="offside-offence">Offside Offence</h2>
<p>Being in an offside position on its own isn’t an offence. However, gaining advantage by virtue of being in that position is. As per the <a href="http://www.fifa.com/development/education-and-technical/referees/laws-of-the-game.html" target="_blank">IFAB Laws of the Game</a>:</p>
<blockquote>
<p>Offence</p>
</blockquote>
<p>A player in an offside position is only penalised if, at the moment the ball touches or is played by one of his team, he is, in the opinion of the referee, involved in active play by:</p>
<ul>
<li><em>interfering with play or</em></li>
<li><em>interfering with an opponent or</em></li>
<li><em>gaining an advantage by being in that position</em></li>
</ul>
<blockquote>
<p>No offence</p>
</blockquote>
<p>There is no offside offence if a player receives the ball directly from:</p>
<ul>
<li><em>a goal kick</em></li>
<li><em>a throw-in</em></li>
<li><em>a corner kick</em></li>
</ul>
<p>Let’s parse this law to understand what it implies. 3 things need to happen at the same time for an offside offence to occur. These are as follows:</p>
<ol>
<li>The player being penalized is in an offside position</li>
<li>The ball touches or is played by a player from the same team as the one in the offside position</li>
<li>The player in the offside position is involved in active play</li>
</ol>
<p>The time of significance is when the ball is played (passed by a player of the same team) and not when it is received by the player who is in an offside position. This is the moment when an assistant referee needs to take a snapshot to make an accurate inference. A player could be one moment in an offside position and within a subsequent moment be in an onside position. The assistant referee needs to keep this in mind and reconsider as appropriate.</p>
<p>What does <strong>the ball touches or is played</strong> mean? It means that the ball is kicked or headed or played is any way that is allowed by the rules of the game. In other words passing the ball by hand, while the ball is in play, wouldn’t count as playing, since that would lead to a “Handling the ball” foul. In addition, three types of restarts (a goal kick, a throw-in, and a corner kick) are explicitly excluded.</p>
<p>Note that the ball <strong>touches or is played by one of his team</strong> and not an opponent. This means a player in an offside position receiving a ball from an opponent is not subject to an offside offence rule.</p>
<p>To quote <a href="http://www.fifa.com/development/education-and-technical/referees/laws-of-the-game.html" target="_blank">IFAB Laws of the Game</a>:</p>
<blockquote>
<p>A player in an offside position receiving the ball from an opponent, who deliberately plays the ball (except from a deliberate save), is not considered to have gained an advantage.</p>
</blockquote>
<p>Watch <a href="https://youtu.be/F5W-bCa-t8Y" target="_blank">this video (from beginning to 1.50 mins)</a> to see real life situations where it’s not an offside offence because the ball is passed to a player in offside position by an opponent. One must also clearly distinguish between a deliberate save by a goalkeeper and a deliberate parrying by a goalkeeper.</p>
<blockquote>
<p>The US Soccer site defines <a href="http://www.ussoccer.com/stories/2014/03/17/12/02/playing-the-ball-in-the-goalkeepers-possession" target="_blank">“Parrying” Versus “Saving”</a> as follows:
The concept of “parrying” is still in the Law; however, parrying is no longer seen at the higher levels of play, because it is no longer an effective tool for the goalkeeper, who has only six seconds to distribute the ball after achieving possession. “Parrying” should not be confused with making a “save.” “Parrying” occurs when the goalkeeper controls the ball with the hands by pushing it to an area where it can be played later. By parrying the ball, the goalkeeper has done two things simultaneously: (1) established control and (2) given up possession. The ball is now free for all to play and the goalkeeper may not play it again with the hands. Referees must watch carefully to see that the goalkeeper does not use a parry (disguised as a “save”) in an attempt to hide the fact that he or she has established possession.</p>
</blockquote>
<p>The third aspect of the offence is that the player in offside position is involved in active play. The 3 types of active play are defined in the guidelines section of the <a href="http://www.fifa.com/development/education-and-technical/referees/laws-of-the-game.html" target="_blank">IFAB Laws of the Game</a> as follows:</p>
<ol>
<li><em>“interfering with play” means playing or touching the ball passed or touched by a team-mate</em></li>
<li><em>“interfering with an opponent” means preventing an opponent from playing or being able to play the ball by clearly obstructing the opponent’s line of vision or challenging an opponent for the ball</em></li>
<li><em>“gaining an advantage by being in that position” means playing a ball</em>
<ul>
<li>i. that rebounds or is deflected to him off the goalpost, crossbar or an opponent having been in an offside position*</li>
<li>ii. that rebounds, is deflected or is played to him from a deliberate save by an opponent having been in an offside position*</li>
</ul>
</li>
</ol>
<p>This means that a player in an offside position not actively playing as per the above definition should not be penalized.</p>
<h2 id="flagging-the-offence--the-indirect-free-kick">Flagging the offence & the Indirect Free Kick</h2>
<p>The assistant referee is in the best position to judge an offside offence. Once he infers an offside offence, he raises his flag and indicates to the referee to stop play. Unless the referee is sure that the offside offence call is incorrect, he blows his whistle and the play is stopped. The assistant referee then indicates the position of the offside offence by the tilt of his flag.</p>
<p>Watch the <a href="https://youtu.be/R7dk8uixjKQ" target="_blank">video on Assistant Referee Signals</a> to learn more about the flag mechanics.</p>
<p>In the event of an offside offence the referee awards an indirect free kick to the opposing team. The indirect free kick is taken from the place where the offside offence occurred.</p>
<h2 id="useful-links">Useful Links</h2>
<p>Next, watch the following videos to reinforce what you have learnt about offside:</p>
<ul>
<li><a href="https://youtu.be/U01-uArNpdw" target="_blank">Offside Rule Explained (in 3 minutes)</a></li>
<li><a href="https://youtu.be/FKTOEHvr3aM" target="_blank">How Offside Works in Soccer</a></li>
<li><a href="https://youtu.be/iND8HoYDL-I" target="_blank">Offside Rule Examples</a></li>
<li><a href="https://youtu.be/DCyP92TVWSQ" target="_blank">Football offside / Law 11 completely explained</a></li>
</ul>
<h2 id="summary">Summary</h2>
<p>Armed with all this information, you should now be able to assess offside position and offside offence correctly. Remember that practice makes a person perfect! The more you actively apply your learnings to real situations, the more you are likely to understand it well.</p>
<p>Would love to hear if you found this post useful or if you found errors in it. Please chime in with your thoughts and comments.</p>tshankyUnderstanding the nuances of Law 11, the offside rule.Big Data Analytics : What’s Next?2013-01-21T16:54:04+00:002013-01-21T16:54:04+00:00http://shanky.org/2013/01/21/big-data-analytics-whats-next<p>While a majority of Fortune 1000 companies are en-route to understanding Hadoop and adopting it in their technology stack, startups in the bay area and elsewhere have started asking the important and inevitable question: “What’s Next?”. Hadoop for the first time has allowed us to analyze massive amounts of data without necessarily indulging in expensive proprietary hardware or software. However, adoption of Hadoop alone isn’t necessarily helping businesses make smarter decisions or unearth completely new facts that could lead to immense growth of top line. The power of scalable infrastructure needs to be supplemented with nifty data mining and machine learning tools, better visualization of results, and easier ways to track and analyze the findings over a period of time. Besides, there is the entire realm of real-time analytics, which is beyond the batch oriented nature of Hadoop.</p>
<p>The “<a title="Global Big Data Conference" href="http://globalbigdataconference.com/" target="_blank">Global Big Data Conference</a>“, scheduled to take place at the <a title="Santa Clara Convention Center" href="https://maps.google.com/maps?q=Santa+Clara+Convention+Center,+Great+America+Parkway,+Santa+Clara,+CA&hl=en&sll=37.269174,-119.306607&sspn=12.35152,25.356445&oq=Santa+Clara+Conven&hq=Santa+Clara+Convention+Center,+Great+America+Parkway,+Santa+Clara,+CA&t=m&z=15" target="_blank">Santa Clara Convention Center</a> on January 28, 2013 answers some of these very important questions around what’s happening in the field of “big data” and what’s to come next. Its a great 1 day conference that has <a title="Global Big Data Conference Agenda Page" href="http://globalbigdataconference.com/agenda.php" target="_blank">a lot of interesting topics, covered by an awesome line-up of great speakers</a>. In order to take advantage of some favorable pricing please <a title="Global Big Data Conference Registration Page" href="http://globalbigdataconference.com/registration.php" target="_blank">register</a> by tomorrow (January 22, 2013) and save a whole <strong>$100</strong>, as compared to the onsite price. In addition, as a reader of my blog, don’t forget to take advantage of the additional 20% discount, which you can avail by using the code: “<strong>SHAS</strong>“. See you all there!</p>tshankyWhile a majority of Fortune 1000 companies are en-route to understanding Hadoop and adopting it in their technology stack, startups in the bay area and elsewhere have started asking the important and inevitable question: “What’s Next?”. Hadoop for the first time has allowed us to analyze massive amounts of data without necessarily indulging in expensive proprietary hardware or software. However, adoption of Hadoop alone isn’t necessarily helping businesses make smarter decisions or unearth completely new facts that could lead to immense growth of top line. The power of scalable infrastructure needs to be supplemented with nifty data mining and machine learning tools, better visualization of results, and easier ways to track and analyze the findings over a period of time. Besides, there is the entire realm of real-time analytics, which is beyond the batch oriented nature of Hadoop.Geolocation in MongoDB at the Silicon Valley MongoDB User Group2013-01-16T09:41:23+00:002013-01-16T09:41:23+00:00http://shanky.org/2013/01/16/geolocation-in-mongodb-at-the-silicon-valley-mongodb-user-group<p>Thanks to all of you, who were able to join me <a title="Geolocation in MongoDB at Silicon Valley MongoDB User Group" href="http://www.meetup.com/MongoDB-SV-User-Group/events/91702142/" target="_blank">at the session last evening</a>. Thanks much for the kind remarks some of you left behind on the meetup message board, post the session. Its very rewarding to know that many of you enjoyed the session and found it very useful. I loved the many questions that were brought up and discussed in the room. Please feel free to send more questions by emailing them to me at st (at) treasuryofideas (dot) com. Alternatively, you could tweet them to me at @tshanky.</p>
<p>The presentation from last evening is available online at <a title="geolocation-in-mongodb" href="http://www.slideshare.net/tshanky/geolocation-in-mongodb-16021143" target="_blank">http://www.slideshare.net/tshanky/geolocation-in-mongodb-16021143</a>.</p>
<div style="margin-bottom: 5px;">
<strong> <a title="Geolocation in MongoDB" href="http://www.slideshare.net/tshanky/geolocation-in-mongodb-16021143" target="_blank">Geolocation in MongoDB</a> </strong> from <strong><a href="http://www.slideshare.net/tshanky" target="_blank">Shashank Tiwari</a></strong>
</div>
<p>For all those who are excited about MongoDB and would like to learn more please join me for “MongoDB in an Hour!” on February 1, 2013. The format of that session would be as follows:</p>
<ul>
<li>1 hour free video session, which will be made available online by or before Feb 1, 2013.</li>
<li>4 half-hour Google+ Hangout sessions for live Q&A.</li>
<li>Unlimited number of Q&A opportunities over email (or over a forum if we create one)</li>
<li>(optional) 1 evaluation exam. Passing the exam would entitle you to a certificate of honor.</li>
</ul>
<p>This 1 hour session is substantially subsidized and I only ask for $25 as suggested donation to cover some of the costs.</p>tshankyThanks to all of you, who were able to join me at the session last evening. Thanks much for the kind remarks some of you left behind on the meetup message board, post the session. Its very rewarding to know that many of you enjoyed the session and found it very useful. I loved the many questions that were brought up and discussed in the room. Please feel free to send more questions by emailing them to me at st (at) treasuryofideas (dot) com. Alternatively, you could tweet them to me at @tshanky.Christmas Trivia2012-12-25T18:17:37+00:002012-12-25T18:17:37+00:00http://shanky.org/2012/12/25/christmas-trivia<div id="attachment_980" style="width: 310px" class="wp-caption alignleft">
<a href="http://shanky.org/2012/12/25/christmas-trivia/screenshot-from-2012-12-25-173617/" rel="attachment wp-att-980"><img class="size-medium wp-image-980 " title="Google's Santa Tracker 2012" alt="Screenshot from 2012-12-25 17:36:17" src="http://shanky.org/wp-content/uploads/2012/12/Screenshot-from-2012-12-25-173617-300x222.png" width="300" height="222" srcset="http://shanky.org/wp-content/uploads/2012/12/Screenshot-from-2012-12-25-173617-300x222.png 300w, http://shanky.org/wp-content/uploads/2012/12/Screenshot-from-2012-12-25-173617-624x461.png 624w, http://shanky.org/wp-content/uploads/2012/12/Screenshot-from-2012-12-25-173617.png 816w" sizes="(max-width: 300px) 100vw, 300px" /></a>
<p class="wp-caption-text">
Google’s Santa Tracker 2012
</p>
</div>
<p>Santa Claus is possibly more popular among kids than the Lord on his own birthday. When and how did the American version of Santa Claus originate?</p>
<p>The American version of Santa Claus comes from the Dutch legend of Sinterklass. Sinterklass is the dutch version of <a title="St. Nicholas" href="http://en.wikipedia.org/wiki/Saint_Nicholashttp://" target="_blank">St. Nicholas</a>, who has the reputation of secretly giving gifts back in the 4th century AD.</p>
<p>The Dutch immigrants brought the concept of Sinterklass to New Amsterdam (modern day New York) in the 17th century. Sinterklass, in his modern version of Santa Claus, initially appeared in the press as “St. A Claus”. In 1809 <a title="Washington Irving" href="http://en.wikipedia.org/wiki/Washington_Irving" target="_blank">Washington Irving</a>, possibly one of most famous New Yorker ever, wrote a book titled the “History of New York” under the pseudonym <span style="font-family: Verdana,Arial,Helvetica;">Diedrich Knickerbocker. You can read that book online: <a title="History of New York by Diedrich Knickerbocker (Washington Irving)" href="http://www.gutenberg.org/ebooks/13042" target="_blank">Knickerbocker’s History of New York, Complete by Washington Irving</a>. In this book Washington Irving described Santa Claus and concocted the legend that he came on a horseback (and not on a sleigh!) each year on the eve of St. Nicholas.</span></p>
<p>In 1823, <a title="Clement Clarke Moore" href="http://en.wikipedia.org/wiki/Clement_Clarke_Moore" target="_blank">Clement Clarke Moore</a> in a poem titled: “A Visit from St. Nicholas”, illustrated Washington Irving’s Santa Claus with additional details that included Santa’s reindeers and his winks and nods. This poem is popularly know as “The Night before Christmas”. Interestingly though Santa himself was an elf as far as this poem goes.</p>
<p>Between 1860 and 1880, <a title="Thomas Nast" href="http://en.wikipedia.org/wiki/Thomas_Nast" target="_blank">Thomas Nast</a>, expanded on these original ideas of Santa Claus and converted him into a fat round character with reindeers, who went every Christmas night giving gifts around the world. His illustration of Santa Claus was published in Harper’s Weekly. A human size and form of Santa Claus as we know of it today, was popularized by Coca Cola in an advertisement in 1931. This advertisement illustration was created by <a title="Haddon Sundblom" href="http://en.wikipedia.org/wiki/Haddon_Sundblom" target="_blank">Haddon Sundblom</a>.</p>tshankyGoogle’s Santa Tracker 2012Geolocation in MongoDB at San Francisco MonogoDB User Group2012-11-30T22:24:02+00:002012-11-30T22:24:02+00:00http://shanky.org/2012/11/30/geolocation-in-mongodb-at-san-francisco-monogodb-user-group<p>Thanks to <a href="http://www.10gen.com/" title="10gen" target="_blank">10gen</a> for having me over at the <a href="http://www.meetup.com/San-Francisco-MongoDB-User-Group/events/87791902/" title="San Francisco MongoDB User Group" target="_blank">last user group meetup in SF</a> to speak on “Geolocation in MongoDB”. I have been leveraging MonogoDB’s geolocation features as a part of the super awesome, yet to be released app: <a href="http://www.doaround.com/" title="doaround" target="_blank">doaround</a>. I shared some of the essentials of geolocation in MongoDB during this session and plan on covering some more at the <a href="http://www.meetup.com/MongoDB-SV-User-Group/events/91702142/" title="Silicon Valley MongoDB User Group" target="_blank">Silicon Valley MonogoDB Meetup in Palo Alto this coming January</a>.</p>
<p>Thanks to all who came for my session. Thanks also for all the great questions and the conversation after the session. Here is the presentation from the session:</p>
<div style="margin-bottom:5px">
<strong> <a href="http://www.slideshare.net/mongodb/geolocation-in-mongodb" title="Geolocation in mongodb" target="_blank">Geolocation in mongodb</a> </strong> from <strong><a href="http://www.slideshare.net/mongodb" target="_blank">mongodb/10gen</a></strong>
</div>tshankyThanks to 10gen for having me over at the last user group meetup in SF to speak on “Geolocation in MongoDB”. I have been leveraging MonogoDB’s geolocation features as a part of the super awesome, yet to be released app: doaround. I shared some of the essentials of geolocation in MongoDB during this session and plan on covering some more at the Silicon Valley MonogoDB Meetup in Palo Alto this coming January.