Reviewer #1

I’m around 3 years into my PhD at this point and so far, I have reviewed for a couple of journals as well as conferences. Mostly journals. But recently I was approached to review for a highly selective conference. And with great paper selectivity comes great hatred towards reviewers.

Of course, I wanted to use this new great power with great responsibility, so I took upon myself, the role of the open minded, somewhat timid, yet trying-hard-to-do-justice, reviewer #1. Which also happened to be my assignment for 4/5 papers that I had to review (I was reviewer #2 for the last paper).

My past experiences with reviewing have been comparatively less challenging. Most of the journal papers that I was asked to review had a good amount of overlap with my own research; plus the review time for journal papers is way more flexible.

Of course, as years have passed I find that my critical thinking abilities have improved, at least in the area that I’m working on. I have cultivated the skill of being able to discern a good idea from a bad one.

Which is all good, but reviewing for this particular selective conference was a whole new challenge. Firstly, the trope. The memes. The reputation. You know what I’m talking about, right?

Well if you’re late to the reviewer’s banquet, there exists a Facebook page with around 22k members at this point, called Reviewer 2 Must Be Stopped! Also brilliantly written blog posts on how not to be reviewer 2. The struggle is real. From my understanding of this meme, there are 2 broad categories of reviewers. Reviewer #1 is the nice, possibly new kid on the block who is kind and appreciative of the paper. Most possibly a frustrated grad student, she is very understanding of the position of the authors. Not too critical but brings up good points. Reviewer #2 (or interchangeably reviewer #3) is the bad cop. The one that puts the authors in deep existential crisis. That makes them question, why they even started pursuing the idea. Reviewer #2 is the dreaded one.

If I wanted to play into the stereotype of reviewer #1, I would have needed to review very responsibly for this conference.

Secondly, I don’t consider my knowledge base to be very broad. I was assigned five different papers with 4 different themes. The themes of course had a fair amount of overlap and I wasn’t allotted a paper that I found myself blanking out on. It was definitely a lot of information to take in, however, I wouldn’t consider that to be a problem. If anything, I only learned from the papers I read. And I did manage to critique a few things based on my understanding, to the extent of my ability. I think reviewing or critiquing, much like anything else, is a skill that one gets better at with practice.

I think reviewing also gave me an insight into how stochastic the process of getting a paper accepted can be. When I told one of my friends about me being a reviewer for this conference, he was taken aback and acted dismissively. He said that this was exactly why the academic reviewing process wasn’t as good as it needed to be. Because of not-so-competent PhD students instead of only well-versed researchers and professors reviewing papers.

I don’t agree. I think it makes sense to have a paper inspected by researchers with various levels of expertise. It’s only good outreach for the paper if it is readable and appeals to a bigger section of the audience. Also, this is exactly how we get trained to become better reviewers. So, I’ll say, reviewer #2, keep doing you. Maybe one day, I’ll be reviewer #2 myself. Until then, I’m happy to get to be good cop and also appear cool on my CV. (I swear, reviewers need more incentives).

As I say all of this, I anxiously await reviews for my own paper submission to the same conference. So here’s hoping I haven’t disappointed of reviewer #2 that much. And so the grind continues.

Meme courtesy http://www.drmalviniredden.com/

Advertisements

The NIPS experience: Newbie edition

Over the past week, I attended (and presented at) one of the biggest conferences in Machine Learning: Neural Information Processing Systems (NIPS) 2017 at Long Beach, California, and the experience was nothing short of exhilarating. There were a number of themes that I made note of, and one blog post is not enough to illustrate them all. So I’ll try to enforce some structural sparsity here to reduce the complexity of this text.

1. NIPS 2017 was humongous.

About 8000 people from academia and industry, thronging to the Long Beach Convention (epi)Center to talk about ground breaking research. It was chaotic. Took me an hour of standing in line to just get my registration badge!

2. Star studded. Both in terms of people and sponsor companies.

3. GANs were an audience favorite. You know something is the new buzz word when companies turn it into a catchphrase and print it on a t-shirt (yes, I did manage to get one for myself!).

You can’t find a better endorsement! There was an entire track of talks specially dedicated to recent advancements in GANs.

4. Bridging the gap between Theory and Practice.

Ali Rahimi’s talk before accepting the Test of Time award was something that was recommended for multiple viewings by multiple people to me. And the entire focus of the talk was about bettering the current brittle algorithmic frameworks, by theoretically analyzing the entire optimization problem, and not treating it like alchemy. There was also an entire workshop dedicated to this theme.

5. “Where’s the party tonight?”

I was asked by at least 5 different people if I was attending a certain sponsor after-party. I had actually got invites to most and RSVP-ed as well, but I found myself extremely exhausted (also, running out of mingling-with-random-strangers stamina). In fact there were people who were particularly interested in the parties and had no clue about what the next talk was about. I guess beyond a point, certain level of sponsor involvement could get worrisome.

6. “Do you want some swag?”

With so many sponsor booths, they had to try different strategies to attract the best minds around. Which meant, flashy sponsor swag (translation: goodies). You could collect enough t-shirts to get through 2 weeks without laundry. These companies certainly know their target, deprived grad students, well.

7. Orals, spotlights and posters.

So much information to gather! NIPS this year, had a record 678 accepted papers, with main themes being Algorithms, Theory, Optimization, Reinforcement Learning, Applications, GANs.

8. Even more orals and posters, in the form of numerous workshops. Also guest appearances from the Women in Machine Learning (WiML) community.

9. Debates and panels.

An interesting debate on the relevance of studying the interpretability problem, sparked a conversation on the various interpretations of the term itself, and whether the problem was motivated well enough, to begin with.

10. A free flow of ideas from every corner.

Some highlights were talks (that I managed to attend) by Bertsekas, Goodfellow, etc. Couldn’t attend some of the morning ones though! And of course, there was a lot to take away from several of the posters sessions. I think I also learnt how to sell an idea better, through my own poster presentation.

I think overall, it was a great learning experience and incredible exposure for a first-timer like me. Hopefully, I will get a chance to visit again! I’m also going to write a part 2 of my experience, which will focus more on some of the more technical ideas that caught my attention at NIPS 2017. Should make a good follow up read after this one! Watch out!

How interpretable is data?

I am finally done with my second semester towards my PhD, which means it’s time for sum-mer and some-more (or a-lot-more) research!

I happened to have two course projects that I only recently wrapped up, and they turned out to be somewhat related! The two topics being sparse principal component analysis (SPCA) and non-negative matrix factorization (NMF). Both of them, key tools to help interpret data better.

So wait. Given a set of data points, can’t we as humans do the intelligible task of interpretation? What do these data-interpretations tools do that we can’t?

The answer: they don’t do anything we can’t. They are just better at interpreting a larger scale of data. They’re like a self-organizing library. The librarian no longer has to assign books to particular sections, the books do that themselves (not that we want to put librarians out of business)!

Those familiar with machine learning will automatically recognize this problem formulation as that of unsupervised learning. Employ algorithms that make sense out of data! Principal component analysis, does just that. It tries to represent the variation in the data in descending order. The first principal direction has the maximum variation in data. Usually the first few principal components (usually, this number is \leq r, where r is rank of the data matrix) are sufficient to explain most of the (variation in) data. Now these “directions” are composed of the relative “importance” of its constituent features.

Mathematically speaking, the PCA problem boils down to the singular value decomposition,

M_{d\times n} = (U\Sigma)_{d\times r} V^T_{r \times n}

where our data matrix M is assumed to lie in a lower dimensional subspace of rank r. Sparse PCA, additionally assumes that the right singular vectors, which are columns of V are sparse. 

The non-negative matrix factorization problem is similar. A non-negative matrix can be decomposed into non-negative matrices W,H,

M_{d\times n} = W_{d\times r} H_{r \times n}

The basic concept utilized in both of these methods is the same: most data has an underlying structure. Imposing the knowledge of this structure should help us extract meaningful information about this data.

Like what? For example in a text dataset, most articles focus on a few core topics. Further, these core topics, can be represented using few core words. This spurred several cool applications, such as detection of trends on social media. In image processing, this has useful applications in segmentation. Representing images as a sum or weighted sum of components. Demixing of audio signals. The list goes on and on and I bet you can already sense the theme in this one.

image02