Figure 1: A simple exercise of coding a RAM module.

Could we do this differently with the linguistic tools like LLMs in 2024? This blog examines different approaches towards classification.

Methodology of Classification

Assigning a HS Code is done hierarchically.

Starting off from Chapters, to Sections to further granularities. If we consider this as a search problem, the search domain keeps shrinking as we continue the “process of selection”. i.e., as soon as we classify a product to be an electrical component, we don’t need to keep Wines in the search space.

A graph could be a great solution to this problem!

Resources at our Disposal

We are focusing on classifying according to the definitions set by Singapore Customs.

• Singapore Trade Classification, Customs & Excise Duties (STCCED) 2022

• Customs Ruling Dataset

• Guide for Classifying Products

Constructing a Graph

STCCED, as the name suggests, contains the hierarchical classification of traded goods. It is a PDF file, and the text is divided into Sections, Chapters and “Subchapters”.

Step 1: ⬇️ Download the STCCED 2022 PDF, use PyPDF2 to extract the text content.

Step 1: Raw text content of STCCED

Step 2: ✂️ Split the text content recursively, into Sections, Chapters and “Subchapters”.

Step 2: The recursively split STCCED!

Step 3: Convert all the subchapters into GraphViz dot files. (Compact representation of parent-child relationships). I used gpt-4 to read the sections and construct these subgraphs.

Super-interesting prompt that converted chapter text into GraphViz

Step 4: Merge the subgraphs hierarchically. The result, is a Graph of 13K+ Nodes, neatly organised!

Zooming into the STCCED 2022 Graph!

Building a Hierarchical Search

For each granularity level (Section, Chapter, Subchapter, Dash and Double Dash), I sent requests to gpt-4 directly with a list of the child nodes. Perhaps this illustration will be of assistance!

Illustration of traversing root -> Section -> Chapter…

Thus, progressive granularity would shrink the total search space down.

Giving it some LLM and Streamlit

For the first build, I used openai directly and got streamlit to create the user-interface.

Try it now on Streamlit:

Demo: 🌏 https://harmony-ai.streamlit.app/
GitHub: 🔗 https://github.com/aniruddha-adhikary/harmony

Limitations and Future Work

1. The current implementation does not go backwards to course-correct. Thus it continues on with a bad classification in a higher granularity.

2. More tokens may be saved by combining vector search along-side graphs for nodes with larger number of children.

Local API Server

Click the [<>] icon to open the API Server pane. Click "Start Server".

OpenAI Environment Variables

Jan AI exposes an Open-AI compatible /chat/completions endpoint. That means, you can convert your existing LLM applications to run on locally hosted models through configuring the OPENAI_API_BASE environment variable.

export OPENAI_API_BASE=http://127.0.0.1:1337/v1

Granted, if your library or application relies on the Python/Node.js openai package.

If you are using Langchain or llama-index, this is enough!

And for the API key, well you don't need one. It can be any random value.

Thank you PyCon TH team!

Overview

This post is a thought-experiment. I would NEVER violate the WhatsApp Terms of Service by using their Web App as an API to automate stuff.

Our goal is to provide our tenant profile for as many property listings as possible (from our list of properties we like), then schedule up viewings.

For the uninitiated, this is the tiring part - constantly providing your profiles, and following up with property agents.

The solution is centered around whatsapp-web.js , it provides a Node.js wrapper around WhatsApp Web. We can initiate conversations or get an event when we receive a reply.

The AI Agent should have access to a few tools, namely,

• get_availabilities(from_date, to_date)

• is_available(time)

• schedule_appointment(time)

This could be possible with some calendar API service, i.e, Google Calendar API with Appointment Slots. (Imagine Calendly)

A simplified flow would look like:

Edge cases to consider:

• Property agent asks for additional information

  • Lease period (1 or 2 years)

  • Demographic Information (Gender, Nationality, Ethnicity, Age)

• Property agent denies a viewing

  • Landlords demographic requirements are not met

• Terminating the Agent and restarting it, it should remember/recall previous conversations.

Providing list of properties

Just providing a list of links to crawl and extract the property listings may not be possible, again due to Terms of Service restrictions, and heavy-handed bot prevention mechanisms of property listing websites.

We could resort to simply keeping a CSV/Sheets file of descriptions of properties, and Property Agents' contact details.

Conclusion

I hope you enjoyed the not at all real, and completely hypothetical project of using an AI Agent to schedule viewing appointments. Act responsibly, within the constraints of the law - in the real world!

Use-case: Generating Classes

We will be generating components (Class) that follow a repetitive yet predictable pattern.

export class HttpBinApi implements ICredentialType {
 name = 'httpbinApi';
 displayName = 'HttpBin API';
}

Notice how many times the developer has to rewrite “HTTP Bin API” in different cases and styles? Moreover, using the wrong case would cause linter issues and never-ending headaches. Could this be automated?

Approach: 🧵 String Manipulation

Simply using JavaScript String functions can take us a long way. Template Strings, joins and possibly a little bit of lodash .

function generateClass(name: string) {
    return [
        `export class ${pascalCase(name)}}Api implements ICredentialType {`,
        `  name = '${camelCase(name)}Api';`,
        `  displayName = '${titleCase(name)} API';`,
        `}`,
    ].join('\n');
}

While this works, it makes me uncomfortable; my mind implicitly sees fragility in this approach. What if the code generator is more complex? What if we need to iterate through arrays or objects? What if the signatures change? It seems unmaintainable.

Hygen

A more practical approach to string manipulation and using templates to generate code — could be Hygen. It handles file creation, manipulation of existing files gracefully.

The devs call it the “The scalable code generator that saves you time.” It is a code generation utility that integrates seamlessly into your development flow. It uses EJS templates to define your file structures.

---
to: src/credentials/<%= h.changeCase.pascal(name) %>.ts
---

export class <%= h.changeCase.pascal(name) %> implements ICredentialType {
    name = "<%= h.changeCase.camel(name) %>";
    displayName = "<%= name %>";
}

Check it out here: Hygen

Manipulating whitespace dependent languages like YAML or Python could be challenging with Hygen. — A dev who tried.

Let’s admit it, in our mind, “code generation is supposed to be a bit more complicated”. The simplicity here may be causing the nerves!

Approach: 🌲 Abstract Syntax Trees

Introduction

What if we could have a more programmatic-way of generating code? Instead of having apparently meaningless string concatenations, we had semantics and meanings?

Introducing ASTs. Code compilers usually do not compile strings of code directly. Instead, it can build up a tree-like structure of branches and objects and conditions.

A tree representation of a simple TypeScript class

Example of AST

For the HttpBinApi class mentioned at the top, the AST representation may look something like this:

  factory.createClassDeclaration(
    [factory.createToken(ts.SyntaxKind.ExportKeyword)],
    factory.createIdentifier("HttpBinApi"),
    //...
    [
      factory.createPropertyDeclaration(
        factory.createIdentifier("name"),
        //...
        factory.createStringLiteral("httpbinApi")
      ),
      factory.createPropertyDeclaration(
        factory.createIdentifier("displayName"),
         //...
        factory.createStringLiteral("HttpBin API")
      )
    ]
  );

Do check out the TypeScript AST Viewer to play around with this Syntax Tree.

Exploring ASTs on our own 🕵🏽

TypeScript AST Viewer in Action

Okay we went from string manipulation to extra-terrestrial programming in a paragraph. I encourage you to use the AST Viewer and click around for now. A few questions to get you started:

1. What does a const look like?

2. How about a let ?

3. A simple console.log ?

4. How about adding two numbers?

5. Okay, looping through an array of numbers?

AST to Code? 👩🏻‍💻

All along, we’ve been converting TypeScript code to AST. But how do we do the opposite? Reconstructing TypeScript code is easy with the help of the built-in ts.Printer .

Step 1: Create a Printer.

const printer = ts.createPrinter({ newLine: ts.NewLineKind.LineFeed });

Step 2: Create an Instance of a SourceFile .

const sourceFile = ts.createSourceFile(
    'HttpBinApi.ts',
    '',
    ts.ScriptTarget.Latest,
    false, // setParentNodes
    ts.ScriptKind.TS,
  );

Step 3: Render as String.

printer.printNode(ts.EmitHint.Unspecified, node, sourceFile);

Check out a fully working, in-the-browser example right here: StackBlitz #1. Do feel free to play around with it!

Where is the Code-Generator? ⚙️

Ah yes yes, now we can assemble everything together into a semi-working code generator.

Step 1: Parametrised the AST representation.

const className = `${_.upperFirst(_.camelCase(name))}Api`;
const classIdentifier = factory.createIdentifier(className);

Step 2: With my trusty Co-Pilot, a large language model — I have refactored the code to make more legible.

Check out the refactored version: StackBlitz #2.

console.log(generateClass('Cloud File Transfer'));

❯ ts-node index.ts
export class CloudFileTransferApi implements ICredentialType {
    name = "cloudFileTransferApi";
    displayName = "Cloud File Transfer";
}

There are no clear-cut winners here. Based on your requirements, you do need to make a call on simplicity vs robustness. Hope you had fun! 🎉

References and Links to Check-out

1. Hygen | Hygen — Code Generation Framework based on EJS Templates

2. Using the Compiler API · microsoft/TypeScript Wiki (github.com)

3. Typescript Code Generation — DEV Community

4. TypeScript AST Viewer (ts-ast-viewer.com)

5. AST explorer

6. TypeScript AST Exploration — StackBlitz

7. TypeScript AST Exploration: Parametrizing — StackBlitz

What is this?

SafeEntry is a simple contact tracing system used by the Government of Singapore as a measure to track the spread of COVID-19. Shops, malls, officers, and other public places put posters out instructing visitors on how to sign in. Those are now available in Bengali!

Download Posters

[Links removed as they are no longer relevant in Singapore]

Introducing generate_series

SELECT * FROM generate_series(1, 5);

generate_series 
            1
            2
            3
            4
            5
 (5 rows)

And, if we want to spice it up by setting the interval, we can do that too! Let's make a list of all the ODD numbers between 1 and 10 (inclusive).

SELECT * FROM generate_series(1, 10, 2);

generate_series 
            1
            3
            5
            7
            9
 (5 rows)

generate_series but with Dates

Let's try just plugging in Date Strings.

SELECT * FROM generate_series('2019-01-01', '2019-01-15');

[42725] ERROR: function generate_series(unknown, unknown) is not unique Hint: Could not choose a best candidate function. You might need to add explicit type casts.

Clearly, something is off. Ah, we didn't cast those values as Dates. Of course. Let's do it.

SELECT * FROM generate_series('2019-01-01'::date, '2019-01-15'::date);

[42883] ERROR: function generate_series(date, date) does not exist Hint: No function matches the given name and argument types. You might need to add explicit type casts. Position: 15

Okay that did identify our value as a date, but it still isn't working. The thing is, we need to provide the interval explicitly. Let's do it!

SELECT *
FROM generate_series('2019-01-01'::date, '2019-01-15'::date, '1 day'::interval);

generate_series     
2019-01-01 00:00:00+06
...
2019-01-15 00:00:00+06
(15 rows)

Alright! That worked like a charm! Of course, let's make it a bit prettier.

SELECT s::date
FROM generate_series('2019-01-01', '2019-01-15', '1 day'::interval) s;

     s      
  2019-01-01
  ...
  2019-01-15
 (15 rows)

Sweet! We can now generate the ranges required.

The Week of Day

I'll let the query and the results talk in this bit.

SELECT
       s::date,
       extract(DOW from s),
       to_char(s, 'day')
FROM generate_series('2019-01-01', '2019-01-15', '1 day'::interval) s;

      s      | date_part |  to_char  
 ------------+-----------+-----------
  2019-01-01 |         2 | tuesday  
  2019-01-02 |         3 | wednesday
  2019-01-03 |         4 | thursday 
  2019-01-04 |         5 | friday   
  2019-01-05 |         6 | saturday 
  2019-01-06 |         0 | sunday   
  ...
  2019-01-14 |         1 | monday   
  2019-01-15 |         2 | tuesday  
 (15 rows)

Starting to make sense now? We can get a numeric value for the day of the week by extracting DOWfrom the interval. And getting the string representation is just as easy as calling to_char. Let's put a weekend constraint now. Check the mapping of days of the week on the table posted above. That's how we got 5 and 6; representing Friday and Saturday.

SELECT s::date,
       extract(DOW from s),
       to_char(s, 'day')
FROM generate_series('2019-01-01', '2019-01-15', '1 day'::interval) s
WHERE extract(DOW from s) in (5, 6);

      s      | date_part |  to_char  
 ------------+-----------+-----------
  2019-01-04 |         5 | friday   
  2019-01-05 |         6 | saturday 
  2019-01-11 |         5 | friday   
  2019-01-12 |         6 | saturday 
 (4 rows)

That's super cool! Now, let's COUNT!

SELECT COUNT(*)
FROM generate_series('2019-01-01', '2019-01-15', '1 day'::interval) s
WHERE extract(DOW from s) in (5, 6);

Voila!

With a Table

SELECT id,
       SUM(CASE WHEN extract(dow from s) IN (5, 6) THEN 1 ELSE 0 END)
FROM tbl
JOIN generate_series(
        tbl.starts_on,
        tbl.ends_on,
        '1 day'::interval) s ON true
GROUP BY id;

So this was my clumsy implementation, that I came up with. I am pretty sure there are better ways to pull this off. Why don't you leave your thoughts and solutions in the comments? And please don't forget to share if you learned anything new! Thanks for dropping by.

You have probably come here seeing errors like:

zipimport.ZipImportError: can't decompress data; zlib not available

WARNING: The Python readline extension was not compiled. Missing the GNU readline lib?
WARNING: The Python bz2 extension was not compiled. Missing the bzip2 lib?
WARNING: The Python sqlite3 extension was not compiled. Missing the SQLite3 lib?
ERROR: The Python ssl extension was not compiled. Missing the OpenSSL lib?

Well, just install these and you’ll be good to go.

$ sudo apt-get build-dep python3
$ sudo apt-get install libreadline-dev libsqlite3-dev bz2-dev libssl-dev zlib1g-dev libffi-dev

You’re welcome.

Getting started

Let's take a look at a run-of-the-mill docstring.

def hello(name):
    """
    Returns a greeting message saying Hello.
    :param name: str
    :return: str
    """
    return 'Hello, ' + name

It's a normal docstring with nothing too special. Let's try to get the doctest module to test our test-less function.

$ python -m doctest -v hello.py

2 items had no tests:
    hello
    hello.hello
0 tests in 2 items.
0 passed and 0 failed.
Test passed.

Well, the output says it clear that we have no tests. Now, let's add a doctest. Remember how the Python Shell or REPL works? Remember how you have three arrows indicating input? Yes. Just copy that style.

def hello(name):
    """
    Returns a greeting message saying Hello.
    :param name: str
    :return: str

    >>> hello('Anam')
    'Hello, Anam'
    """
    return 'Hello, ' + name

Time to run the doctest module again.

$ python -m doctest -v hello.py

Trying:
    hello('Anam')
Expecting:
    'Hello, Anam'
ok
1 items had no tests:
    hello
1 items passed all tests:
   1 tests in hello.hello
1 tests in 2 items.
1 passed and 0 failed.
Test passed.

Ah, that looks more like it.

Expecting Exceptions

Let's try something more fun. Let's add a case in our function, if the provided name is just composed of digits, we refuse to greet digits!

def hello(name):
    """
    Returns a greeting message saying Hello.
    :param name: str
    :return: str

    >>> hello('Anam')
    'Hello, Anam'
    """

    if name.isdigit():
        raise ValueError('We do not greet numbers')

    return 'Hello, ' + name

Now, how do we test for exceptions? Well, remember the mantra.

Replicate your shell; success you will be showered with upon you.

So, just replicate your shell! Let's see how the shell reacts to digits.

So, in the doctest,

def hello(name):
    """
    Returns a greeting message saying Hello.
    :param name: str
    :return: str

    >>> hello('Anam')
    'Hello, Anam'
    >>> hello('121')
    Traceback (most recent call last):
    ...
    ValueError: We do not greet numbers
    """

    if name.isdigit():
        raise ValueError('We do not greet numbers')

    return 'Hello, ' + name

Just the first line and the last line of the exception. Don't confuse doctest with a complete, well-formatted and fully laid out stack trace, please! And just three dots to indicate there were more to it. So, let's run this again.

$ python -m doctest -v hello.py

Trying:
    hello('Anam')
Expecting:
    'Hello, Anam'
ok
Trying:
    hello('121')
Expecting:
    Traceback (most recent call last):
    ...
    ValueError: We do not greet numbers
ok
1 items had no tests:
    hello
1 items passed all tests:
   2 tests in hello.hello
2 tests in 2 items.
2 passed and 0 failed.
Test passed.

And yes.

In Production

Of course, in production, you do need to use a test runner. Something in the category of pytest or nose. I personally prefer pytest. Let's take a look at configuring pytest for this.

Not so surprisingly, it discovered no tests in the default configuration. We need to add a simple flag to it --doctest-modules .

That seems to be working, let's try to intentionally run our tests to check if it is really working.

And that is working perfectly. To avoid putting the --doctest-modules flag all the time, consider making a tox.ini file to place your pytest configuration. More on this in the pytest documentation.

%timeit

%timeit does exactly what it sounds like. It times your function. It niftily (is that a word?) providers the standard deviation of multiple runs as well. So essentially you probably won't get lucky by chanceperformance reports.

%prun

This is a rather comprehensive profiling. You can check the functions of being called, the number of times they're called, and so on. This is especially useful while writing something recursive or with a deep call stack.

If you are using third-party plugins or modules, often it is difficult to pinpoint the performance issue - in your code or in their code? %prun will help by inspecting the entire call stack.

Of course, while writing recursive functions, you can actually visualize how your number of calls grow out of proportions as the problem size increases!

Cheers!

TL;DR

1. When your application bootstraps, build combinations of all the possible states that might result in different content among the views.

2. Generate the actual content for all those combinations generated above.

3. Store the generated content as static files, serve with nginx.

4. Proxy the non-static requests directly to the application.

Depending on the design of your application, you probably will see a considerable number of requests getting served by nginx directly without any involvement of your backend application.

To Cache, or not to Cache?

You probably cannot staticify everything in your application. You have to choose what needs to stay static and what needs to remain dynamic. A simple rule of thumb is volatility.

• If new records/data/rows get added frequently, that's a big no-no for staticifying it.

• If the number of possible values is very high for any parameter you wish to base content generation upon, that is also a big no-no.

• You cannot use static sites for validation and processing of a form.

Imagine you will generate static content based on a user's account type and some other parameters.

account_type (free, basic, premium, gold, platinum)
billing_type (wirecard, credit card, nets, ezlink)
is_beta_user (yes, no)

So, you ran into 5 x 4 x 2 = 40 possible states. So the math is easy. Things can go out proportions unless you are careful! So carefully generate and manage the combinations.

Generation

Simply think of this passing the context variables to a templating engine for it to render an output. That's it. You got it.

Hyperlinking and Directory Structures

One approach I personally prefer is to create directories for each possible value combination. So it would look like,

free/index.html
free/upgrade.html
premium/index.html
premium/wirecard/billing.html

Entry

The application should have a dynamic entrypoint, from where the user is redirected to the appropriate static directory.

I just did a quick write-up, not sure if this will spark anyone's interest. If you want to know more about this, please leave a comment so that I can get to know that I have to write another followup!

(I used this approach to build a USSD service, where authentication doesn't need to be performed by the application. So yeah that's why it was practical!)

cURLing

cURL is a great tool for sending HTTP requests. Let's try to send one.

curl http://httpbin.org/ip

Hmm, okay it can do stuff. Let's make it a bit more verbose shall we? Just add a -v flag.

curl -v http://httpbin.org/ip

Ah now things are interesting. Do you see a few lines starting with '>'? Well those are the headers that are SENT to the server. And the lines starting with '<'? They are response headers that are received. The last part is the content itself. Of course, the < and > are there for representation only, they are not transmitted.

Telneting

telnet is a great tool to make raw socket connections and really dissect almost any TCP based protocol like HTTP. Lets try this.

telnet httpbin.org 80

Afterwards, type in a raw HTTP request yourself!

GET /ip HTTP/1.1
Host: httpbin.org

Now, press enter or return twice!

Who needs Firefox, Chrome or cURL! Looks like Telnet is all we need ;) Okay enough jokes for now. This is serious business. You see in the end HTTP is just a plain socket connection that has some specific "lingo" to send requests and receive responses. (Protocol, duh).

Dissecting the Request

Let's see how exactly we made the request.

GET /ip HTTP/1.1
Host: httpbin.org

Pretty straightforward. The HTTP method in caps, the path and the HTTP version in the first line. After that, the request headers follow. In our case we just mentioned the 'Host' header to play safe with VirtualHost-enabled systems.

Back in the days, running a website was expensive because you needed an entire box (a server computer) to host a single website. Then we invented a way to serve multiple websites from the same IP address / machine with virtual hosts in the Apache Web Server.

The internet as we know it exists for this 'Host' header.

The two newlines indicate we are done with our request, and the server can parse it and send back a response.

Let's be bad

Let's send a malformed HTTP request (muhahaha).

telnet httpbin.org 80

Afterwards, type in a malformed request yourself!

muhahahaha

Now, press enter or return twice!

Yup :( It was a Bad Request. We'll write our own HTTP server with plain old sockets soon. Till then, why not explore the exciting world of HTTP response headers on Google ;) Yeah and take a look at RFC 2616.

So I decided to put together a series of articles on NLP for developers. Thus I skipped mathematics, calculus and analytical geometry along with rocket science unless absolutely necessary. The series will focus on applications, rather than theories. However, I do encourage readers to get a technical understanding of how and why things work, the way they do. Let’s get started.

The Setup (for now)

First of all, we will get our feet wet with holy NLP water through our friend TextBlob. It is a Python library for NLP. Why Python? Because Python is very popular for the purpose of NLP, AI, ML and scientific computing in general. TextBlob has a very easy and expressive API, making it easy for others to grasp and use.

You have probably heard of NLTK and how it saved the world from impending doom. So why are we not using NLTK? We are. TextBlob is a wrapper over NLTK, abstracting away many fine-tuned details. We are not here to get intimidated by NLTK, we are here to experience NLP in action.

So, we will be needing:

• A computer running Mac OS X, BSD or a variant of Linux (I don’t know how things work in Windows)

• Working knowledge of Python 2.x or 3.x

• Gallons, Buckets, Containers or entire ship shipping ship full of patience

Start off by installing TextBlob.

pip install textblob

We are done (for now).

Hello, NLP!

Let’s get into processing some sentences.

from textblob import TextBlob

text = "Aniruddha did not attend class because he was sick. The lecturer marked him absent."
blob = TextBlob(text)

Okay, let’s break things down. The first line imports a class, nothing fancy. The second line declares a string, nothing fancy. Ah, the third line - we are creating a TextBlob object from our text string. Our blob of text is now ready for some cool NLP! Let’s figure break the whole text down into sentences.

Breaking Down (aka Tokenization)

Let’s try this. (BTW, we are on the Python Shell)

blob.sentences

Output:

[Sentence("Aniruddha did not attend class because he was sick."), Sentence("The lecturer marked him absent.")]

There are two sentences as we can see, broken down or tokenized just as we wanted. Let’s try something more granular, let’s break down the Sentences into Words.

blob.sentences[0].words

That worked out quite well! Let’s get back to some grammar lessons from elementary school.

Parts of Speech Tagging

Parts of Speech (PoS) are the individual words of a sentence, they are classified into categories like nounds, pronouns, adjectives, adverbs, prepositions etc. Identifying PoS is an essential task in NLP. Let’s see how it all works.

blob.sentences[0].pos_tags

Output:

[('Aniruddha', u'NNP'),
 ('did', u'VBD'),
 ('not', u'RB'),
 ('attend', u'VB'),
 ('class', u'NN'),
 ('because', u'IN'),
 ('he', u'PRP'),
 ('was', u'VBD'),
 ('sick', u'JJ')]

Weirdo acronyms? Well, ('Aniruddha', 'NNP') wants to tell us that, Aniruddha is an NNP, AKA a singular proper noun. Similarly, JJ indicates an adjective. How do I know?

import nltk

nltk.help.upenn_tagset('NNP')

Now you can figure out the mysterious acronyms as well. Let’s try some transformations.

Transformers!

Let’s try something even cooler. Remember our sentence[1], Sentence("The lecturer marked him absent.")? Let’s try to make lecturer plural.

lecturer_word = blob.sentences[1].words[1]
lecturer_word.pluralize()

There are some similar operations in TextBlob, try them! Read the documentation.

Synonyms and related words

You could just build a very simple Dictionary of Synonyms when you have a Python interpreter. Let’s take a look at the synonyms of marked.

marked_word = blob.sentences[1].words[2]
marked_word.get_synsets()

[Synset('tag.v.01'),
Synset('mark.v.02'),
Synset('distinguish.v.03'),
Synset('commemorate.v.01'),
Synset('mark.v.05'),
Synset('stigmatize.v.01'),
Synset('notice.v.02'),
...]

That will enough for this part. Let me know in the comments if you liked TextBlob, and/or whether you want more stuff of NLP. Thanks for dropping by.

What is LaTeX and why does it matter?

Wikipedia prescribes LaTeX as a document preparation system; the official website states it as a typesetting system. I would describe LaTeX as a system for preparing documents where fine-grained pixel-by-pixel perfection is a reality. The leading conferences or journals provide their template in LaTeX, and often in word-processor formats. Personally, I have found word processor behavior to be quite unreliable and prefer LaTeX for it’s ease and simplicity. A quick search on your favorite search engine might (and probably will) more reasons to use LaTeX.

BibTex is a markup format for bibliography generation. It is used a lot in par with LaTeX for generating citations according to the required styling.

ShareLaTeX

ShareLaTeX is a website similar to Google Docs, for LaTeX. It allows collaborative editing, history tracking and free hosting online. It also helps out by providing a plethora of templates to choose from; including templates from well-reputed journals or conferences. You will not need to install anything other than a modern web browser on your computer for using ShareLaTex, everything is done online. As per expectation, it can preview and export as PDF.

CiteThisForMe

CiteThisForMe is a great website for managing your bibliography. You can generate citations for books, websites, journal articles and more. By the way, citations are the references you make to content your paper “borrows ideas from”. A variety of export styles are supported, including Harvard, MLA, Vancoover et cetera or the generic BibTex.

Google Scholar Extension for Chrome

This extension lets you search for any piece of text on Google Scholar. Also lets you discover papers, generate citations all without leaving your current tabs. I find it extremely handy for generating quick citations and searching for referenced papers while reading.

I am planning to write down a very basic tutorial on LaTeX syntax sometimes in the near future. Please let me know in the comments if you are interested.

1. Download the Official Developer ROM

2. Visit http://192.168.31.1, login and manually upgrade to the Official Developer ROM. Ignore any warning regarding downgrading.

3. Wait for 5-7 minutes. After the router is back on again, you need to set a root password to be able to log in. Login to the router and copy the “stok” but from the address bar. Open up a shell and,

curl -d "oldPwd=your_admin_pass&newPwd=desired_root_pass" \
"http://192.168.31.1/cgi-bin/luci/;stok=<stok from browser url>/api/xqsystem/set_name_password"

1. Now a response should indicate a code. If the code is set to 0, you have successfully set your password. Otherwise if you receive a text in chinese, translate that to figure out what is wrong.

2. Happy Hacking!

Install Cygwin

According to the website of Cygwin, Cygwin is “a large collection of GNU and Open Source toolswhich provide functionality similar to a Linux distribution on Windows.” It provides a decent terminal interface with the GNU / Unix tools you love and crave for. You should download the 32-bit (setup-x86.exe) or 64-bit (setup-x86_64.exe) installer according to the configuration of your computer.

Open the installer and follow the on-screen instructions, on the package selection screen, make sure to install at least zsh, git, nano and wget alongside the default selections. Once the installation is complete, you will find a Cygwin Terminal in your Start Menu (Is it still called start?).

[Note: Install the Cygwin git even if you already have Git for WIndows installed.]

Set zsh as the default shell

Open up Cygwin Terminal. Cygwin does not have chsh for changing shells, nor does it have a /etc/passwd file. Thus, we need to make one (Thanks BrunoLM).

mkpasswd -l -p "$(cygpath -H)" > /etc/passwd

Open the /etc/passwd file for editing.

nano /etc/passwd

You will be presented with a gigantic text file. Find the line that begins with your username. Go the the end of that line, and replace /bin/bash with /bin/zsh.

Install oh-my-zsh

oh-my-zsh is a configuration (+ theme / plugin system) for the Z Shell. The oh-my-zsh installation script should be fetched and executed through the “wget method”, since we did not yet install curl. So, in the Cygwin Terminal and paste the following,

sh -c "$(wget https://raw.githubusercontent.com/robbyrussell/oh-my-zsh/master/tools/install.sh -O -)"

zsh should be installed and ready for configuration.

Solarized Theme and Powerline fonts

Karlin Fox ported the Solarized theme to MinTTY (the terminal which comes by default with Cygwin), and it is available at karlin/mintty-colors-solarized.

git clone https://github.com/karlin/mintty-colors-solarized.git ~/.solarized
echo 'source ~/.solarized/mintty-solarized-dark.sh' >> ~/.zshrc # for light, replace dark with light

Some fancy zsh themes require powerline-patched fonts. Powerline patched versions of the most popular mono fonts are available at powerline/fonts. Install the ones required after cloning / downloading (zip) the repository. Make sure to change the font of MinTTY.

Right click the title bar of the Terminal, click Options. Change your font to a Powerline patched font you installed.

Install apt-cyg

apt-cyg acts similar to apt-get, works great for installing packages! Installation is fairly straightforward,

lynx -source rawgit.com/transcode-open/apt-cyg/master/apt-cyg > apt-cyg
install apt-cyg /bin

That should help you get started!

git clone https://github.com/mozilla-b2g/B2G.git
cd B2G
./config.sh dolphin

The resulting images will result in a device showing a white bootscreen if flashed to a GoFox F15. Why? If you plunge down into this repository, you will find a lot of target devices to build for. scx15_x3542plus is our target device, this denotes our Symphony GoFox F15. However, the device scx15_sp7715ga is selected by default, causing the conflict. To resolve this, just do a little search and replace in the config.sh file.

sed -i 's/scx15_sp7715ga/scx15_x3542/g' config.sh

Now, we are ready to build Firefox OS for GoFox! Make sure to use the v1.4 or v2.1 branch to build, upto writing the blog posts, the other branches failed to boot on GoFox F15 in my experiments.

BRANCH=v1.4 ./config.sh dolphin

And then finally make sure you have something to do while you wait for the very lengthy build process to complete, and issue a build.

./build.sh

And you will have your images (boot.img, recovery.img, system.img, cache.img, userdata.img) ready in out/target/product/scx15_x3542 to be flashed via fastboot