Reviewer #1

I’m around 3 years into my PhD at this point and so far, I have reviewed for a couple of journals as well as conferences. Mostly journals. But recently I was approached to review for a highly selective conference. And with great paper selectivity comes great hatred towards reviewers.

Of course, I wanted to use this new great power with great responsibility, so I took upon myself, the role of the open minded, somewhat timid, yet trying-hard-to-do-justice, reviewer #1. Which also happened to be my assignment for 4/5 papers that I had to review (I was reviewer #2 for the last paper).

My past experiences with reviewing have been comparatively less challenging. Most of the journal papers that I was asked to review had a good amount of overlap with my own research; plus the review time for journal papers is way more flexible.

Of course, as years have passed I find that my critical thinking abilities have improved, at least in the area that I’m working on. I have cultivated the skill of being able to discern a good idea from a bad one.

Which is all good, but reviewing for this particular selective conference was a whole new challenge. Firstly, the trope. The memes. The reputation. You know what I’m talking about, right?

Well if you’re late to the reviewer’s banquet, there exists a Facebook page with around 22k members at this point, called Reviewer 2 Must Be Stopped! Also brilliantly written blog posts on how not to be reviewer 2. The struggle is real. From my understanding of this meme, there are 2 broad categories of reviewers. Reviewer #1 is the nice, possibly new kid on the block who is kind and appreciative of the paper. Most possibly a frustrated grad student, she is very understanding of the position of the authors. Not too critical but brings up good points. Reviewer #2 (or interchangeably reviewer #3) is the bad cop. The one that puts the authors in deep existential crisis. That makes them question, why they even started pursuing the idea. Reviewer #2 is the dreaded one.

If I wanted to play into the stereotype of reviewer #1, I would have needed to review very responsibly for this conference.

Secondly, I don’t consider my knowledge base to be very broad. I was assigned five different papers with 4 different themes. The themes of course had a fair amount of overlap and I wasn’t allotted a paper that I found myself blanking out on. It was definitely a lot of information to take in, however, I wouldn’t consider that to be a problem. If anything, I only learned from the papers I read. And I did manage to critique a few things based on my understanding, to the extent of my ability. I think reviewing or critiquing, much like anything else, is a skill that one gets better at with practice.

I think reviewing also gave me an insight into how stochastic the process of getting a paper accepted can be. When I told one of my friends about me being a reviewer for this conference, he was taken aback and acted dismissively. He said that this was exactly why the academic reviewing process wasn’t as good as it needed to be. Because of not-so-competent PhD students instead of only well-versed researchers and professors reviewing papers.

I don’t agree. I think it makes sense to have a paper inspected by researchers with various levels of expertise. It’s only good outreach for the paper if it is readable and appeals to a bigger section of the audience. Also, this is exactly how we get trained to become better reviewers. So, I’ll say, reviewer #2, keep doing you. Maybe one day, I’ll be reviewer #2 myself. Until then, I’m happy to get to be good cop and also appear cool on my CV. (I swear, reviewers need more incentives).

As I say all of this, I anxiously await reviews for my own paper submission to the same conference. So here’s hoping I haven’t disappointed of reviewer #2 that much. And so the grind continues.

Meme courtesy http://www.drmalviniredden.com/

Advertisements

Optimization in Life

I’ve been doing a lot of optimization related work and courses, for my PhD, most notably in convex optimization and non-linear programming. They say that the best way to learn theory is to implement it in real life, and so I thought that it wouldn’t hurt to find ways to optimize… life… eh? On that optimistic enough thought, here we go:

1. The steepest descent is not necessarily the fastest. A common thing that people do when they are in an unwanted situation is to do starkly opposite of what they were initially doing, i.e. $-\nabla f(x)$. This seems to be a go-to solution for minimizing conflict. However, it is well known that to reach the point of minimum (conflict), steepest descent can take far more number of iterations than other gradient based methods. So take it easier, guys. Extremeness is not a smart option.
2. When bogged down by multiple issues, solve one-problem at-a-time. Coordinate descent is an approach in which the objective (life’s problems) is minimized w.r.t a fixed coordinate at a time. It’s known for its simplicity of implementation.
3. The apple does not fall far from the tree. So when Newton came up with his method for optimizing functions, the initial estimates did not fall far from the optimum, most notably in the case of quadratic functions. Turns out, it helps to approximate functions at each point with quadratic estimates, and then to minimize that quadratic estimate. Basically, take a problem and convert it into an easier sub-problem that has a known minimum. Move on to the next sub-problem. This fetches you the global optimum much faster.
4. While positive definiteness is ideal, positive semi-definiteness is good too. If the Hessian of the function to be minimized is positive semi-definite, then the function is convex and can be minimized easily (its local optimum is the global optimum). So keep calm (and kinda positive) and minimize issues.
5. Often when there are too many parameters to handle, we tend to overfit a fairly complicated model to our life. In such cases, it is a good idea to penalize over-complication by adding a regularization term. Regularization also helps in solving an ill-posed problem. If we tend to focus on only a specific set of problems, we forget other facets of life, which leads us into making poorer choices. The key is to find the right balance or trade-off.
6. Some problems actually have closed-form or unique solutions. There’s just one possible answer which is apparent enough. In that scenario the optimal strategy should be to stop optimizing. Stop contemplating, just go-get-it!
7. On a closing note, heuristically speaking, one would need to try out a bunch of optimizing techniques to find the optimal optimization technique.

XKCD

To make this post even more meta, how optimal would it be if the moral of this post converged to this statement?

Why Murphy was probably right

So, there’s this law by Murphy that most of you must be aware of.

If anything can go wrong, it will.

Now, the origin of Murphy’s law is quite well explained here.

And so goes the original Murphy’s law:

If there are two or more ways to do something, and one of those ways can result in a catastrophe, then someone will do it.

Now the situation that gave rise to this quote is something like this.

Edward A. Murphy, Jr. was one of the engineers on the rocket-sled experiments that were done by the U.S. Air Force in 1949 to test human acceleration tolerances (USAF project MX981). One experiment involved a set of 16 accelerometers mounted to different parts of the subject’s body. There were two ways each sensor could be glued to its mount, and somebody methodically installed all 16 the wrong way around. Murphy then made the original form of his pronouncement, which the test subject (Major John Paul Stapp) quoted at a news conference a few days later. (Source)

I’d think the odds of failure were quite high. How?

The person in charge of installing the accelerometers can be called Mike. Why? It’s a standard enough name. Now, Mike probably wasn’t a smart enough guy.

1. He did not know which side of the sensor went where and randomly installed all accelerometers, using no common sense, failing to set the right combinations = $0.5 \times (1 - 0.5^{16})$ [FAIL]
2. He did not know which side went where and randomly installed all accelerometers, using no common sense  but luckily fixing the right combinations = $0.5 \times (0.5^{16})$ [SUCCESS]
3. He had the sides interchanged and installed all in the same way. Well, at least he had some common sense to install all in the same way = $0.5 \times 0.5$ [FAIL]
4. Mike was smart. He got the sides right and had the sense to install all in the right way = $0.5 \times 0.5$ [SUCCESS]

Let’s give Mike the benefit of doubt. Maybe he was smart. Let’s assign that a probability of $0.5$. Probability Mike was dumb is $0.5$.

The probability of failure is then approximately $0.75$. If you think of any ideal situation too, the probability of the chain of events leading to a success, when multiplied, is quite low.

Let’s look at it this way. The event: Me getting a sound night’s sleep. Shouldn’t be hard right?

Why it doesn’t work: I have a roommate who keeps talking loudly on the phone till wee hours. Why would I have a roommate? I am a research assistant, we don’t get paid well enough for me to be able to afford a better room. Why am I a research assistant? I want to do a PhD. Why do I want to do a PhD? You get the drift.

Turns out, I was almost destined to have a painful right ear, being subjected to continuous loud mindless rants in the middle of the night. The consequences of a lot of our actions aren’t really predictable until events transpire in due course of time. But when they do happen, it’s not that hard to chart out the trajectory of what might have caused them. And so, if anything wrong can happen, perhaps your brain is able to trace that trajectory in advance to forecast what will go wrong.

Here’s the catch though. When things are expected to go wrong and they don’t, we are so happy with the outcome, we barely recognize it as a failure of the law. So Murphy was a genius, in framing a law whose exceptions would go easily unnoticed. Whoever thought that something so iconic would come out of so much pessimism?

Then again, as Phil Dunphy from Modern Family would say…