Last week at work was pretty interesting. On Tuesday, L and I flew up to Dayton for the day to meet with our sponsors. That meant a 6am flight out and originally a 11pm return flight. Fortunately, we wrapped it up a bit early and landed home back around 8:30pm. The meeting went well. We were presenting some ideas that I had to the sponsors and they seemed to like them. Strangely enough, in talking to L, he said that he hadn’t really understood what I was proposing until the meeting. Not bad for a guy who wrote a short paper for the sponsors that described what he didn’t understand! But then, I guess that’s why L’s a full professor 🙂
Wednesday was largely recovery. Thursday, I met with the other modeling guy on the team who was in from New York. We discussed what I wanted to do and he seemed to have a good handle on it which was good because later in the afternoon L came in and wanted to discuss it in more detail. He had been stuck in an airport and had time to think through some things. The bad news is that he didn’t think my proposed modeling would work. The good news is that he thought he had a fix.
Turned out that I had forgotten that the underlying model we are using is a Markov model. The basic requirement of the model is that the software agent can only be in one state at a time. I was essentially proposing that the agent could be in multiple states simultaneously. As we talked more, I realized that L’s proposal was to generalize what I was getting at. Essentially, I described the special case of m=1. Before my proposal, we had been working with m=n. L’s proposal was to let m vary between 1 and n. This bugged me at first since I thought m>1 wasn’t necessary. I was also concerned since L and the other modeler, N, had theoretical objections to m=1. I pointed out that really for any m<n, their theoretical objections held true, so nothing was really more wrong with m=1. Everyone agreed with that and we further agreed that a varying m would still have some theoretical problems, but would be more effective, more realistic and more tractable than m=n or m=1.
Wrapping that up, I received a short email from L on Friday thanking me for the work, saying that was a good white board session and that he could see how my thinking was influencing his for the better. That was nice 🙂
p.s. if y’all are really nice, I promise not to write a blog post on Markov decision processes
  