This post is about how to get a simple list of Twitter Followers/Following that you can work with, using the data that’s already available for you – on your own browser.  It’s a concept similar to how bookmarklets work, but this approach doesn’t involve installing anything on your machine.

This quick solution runs on your own Chrome browser,  requires no API dev work, requires nothing to be installed, requires no additional page requests, and requires no 3rd-party services. It’s very simple, you control it, and it uses your browser’s own inspection tools. It’s just Javascript using XPath to navigate the page’s DOM.

1. Use Chrome (it might work with other browsers, but haven’t checked);
2. Go to the followers/following section and scroll all the way to the bottom so they’re all visible on the page;
3. Right click the web page and choose “Inspect”;
4. You’ll see something like what I’m showing here. Click on Console; 
5. Paste the following Javascript code on the prompt and press Return/Enter. Note that it’s one very long line;

a=$x("//div[@class='ProfileCard-userFields']");b=[];for(var i=0;i<a.length;i++){b.push("@"+$x(".//span[@class='u-linkComplex-target']/text()",a[i])[0].textContent+"\t"+$x(".//a[contains(@class,'ProfileNameTruncated-link')]/text()",a[i])[0].textContent.replace(/(^\s+|\s+$)/g,''));b.join("\n")}

As a result, you get a newline-separated list of handles and names that can be easily copied, and then pasted into a text editor or spreadsheet. Handles and names are separated with tabs so you can easily separate the data into columns for filtering and/or sorting.

This seems to work fine today… but if the design of the page loaded by your browser changes, this will likely stop working.

Back in 2011-2012 I put a lot of time and energy into creating a simple and sleek JSON API framework for quick intelligence prototyping; an API capable of managing JSON objects, and performing a lot of smart computing tasks. Fast forward to 2016, I decided to open source the codebase, sharing it with the world because I believe this framework, although a bit outdated by now, still has the potential to help others.

SQLpie™ is an open source API framework that uses all sorts of SQL statements to creatively perform all kinds of computing tasks (thus, SQLpie). With SQLpie, developers can store JSON objects in a SQL database and run a lot of information retrieval and machine learning tasks on the data, covering areas such as: Text Classification, Text Summarization, Collaborative Filtering (item recommendation and similarity), Boolean/Vector Search, Document Matching, TagClouds, etc… The project is 100% written in Python and runs on top of a MySQL database.

The SQLpie project went after a lot of big challenges, and although I do not advocate that it includes the best implementations to handle all of those tasks, I believe the combined effort can help people quickly prototype new ideas, and hopefully, create new and awesome products.

Its API services can help developers with the following type of questions:

• How can one store JSON documents? (answer: documents services)
• How can one keep track of document relationships? (answer: observations services)
• What documents exist for query Q? (answer: indexing and search services)
• What documents are located near location L? (answer: geosearch service)
• What top keyphrases and keywords relate to query Q? (answer: tagcloud search service)
• What are the key sentences, entities, and terms associated with document D? (answer: summarization service)
• What documents are similar (or relate) to document D? (answer: document matching service)
• Will user U like document D? (answer: classification service)
• How likely is user U to like document D? (answer: classification service)
• What documents is user U likely to love based on user data? (answer: recommendation service)
• What other users have a document taste similar to user U? (answer: similarity service)

If you’re a developer, learn more at SQLpie.com. The project is hosted on Github.

Cheers,
~ Andre Lessa

Let’s start with the kind of question you are likely to ask yourself the first time you come across something new.

“What do I need a Benchmarking Engine for?”

A possible short answer is this: To efficiently and automatically identify opportunities for business performance improvement, customer/vendor satisfaction, and revenue generation.

Now for a more comprehensive answer… Continue reading →

Here’s another interesting problem that I solved. This one relates to SQL Server.

The problem:

To write a single database query that would allow me to get multiple row counts depending on certain pre-defined conditions. Let me make it clear … I had a table called nodes and I needed to count how many times certain non-unique records had been saved. Since the table was huge in size, the idea of using multiple queries scared me so using a single query to solve the problem was the way I found to optimize the performance and the algorithm.

The solution:

Many developers are used to write statements like select count(*) from certaintable where tablecolumn = specialcondition … that works great when you just need to count one thing at a time. My solution was to approach the problem by moving the where condition to the select section of the statement.

The query:

select
sum(case when node_id < 300 then 1 else 0 end),
sum(case when node_id > 200 then 1 else 0 end),
sum(case when node_id between 200 and 300 then 1 else 0 end)
from node

The recipe:

The secret sauce was to use the sum/case combo instead of the standard count function. By testing each condition I wanted with a case statement, and adding up the number of times each condition turned out to be true (using the sum function), I was able to achieve my goal.

Just recently I came across this request to shift an entire slice of an array while keeping the algorithm cost to its bare minimum. Let’s put it this way. It’s pretty simple if you have something like ABCDE and you want to shift the elements so it becomes CDEAB. Now, the big problem is this – what if you have 1 billion bytes in this array and you need to perform sort of the same operation? That’s trick, right? Most likely, I wouldn’t be able to afford an additional high amount of memory to perform this.

So here’s what I came up with. The code is written in Python.

def f_ArrayExercise(a, i):
    n = 0
    while i <= len(a) - 1:
        a[i], a[n] = a[n], a[i]
        i += 1
        n += 1
        if len(a)%2 and len(a) > 1:
            a[len(a)-1], a[len(a)-2] = a[len(a)-2], a[len(a)-1]
    print a

The secret here is to swap one element at a time in order to save in memory space. However, note that with minor modifications we can easily swap pre-determined chuncks of the array at a time instead of limiting ourselves to one element – just in case we know we can afford a couple more elements. That would help optimize the processing time.

The first argument of the function is the array itself, the second argument is the exact spot you want to use to start shifting the array.

When you play around with this function, you get something like this: