Thursday: Q128/Waves

Not cumulative.

Monday: That doesn't make sense.

I was able to get it to partially work, but the thing I've been avoiding at work while reading up on the code still isn't finished.  I can't figure out how it's doing a mapping, and I don't see any place that's defining the mappings it is getting right.

Tonight was also board game night, which I came out of with a pretty decent 50/50 split.  I could have won one of the games of Sabotage if I'd waited slightly longer to betray everyone else.

It's muggy and hot, but I should probably make this again once it gets cold.  I think some sort of pork would help flesh it out into more than just a side.

• This was my local grocery store.
• Yeah.
• So Baylor is just the absolute worst.
• Am I sufficiently not lazy to do the math for this?
• From this data, you can assume that this is reasonably described by a three-component mixture model (using Gaussians because I'm lazy and they're good enough).
• Feeding the individual values to my GMM code yields all modes with approximately sigma = 1, and means and mode fractions:
• F: m = (1.89, 6.74, 9.12); P = (0.04, 0.49, 0.46)
• M: m = (1.36, 6.28, 8.85); P = (0.23, 0.56, 0.21)
• The bias for males is then something like a half point.  It's smaller for the highest ranked mode, but that might just be because of the fact that the range and sampling is limited.

Friday: it's always best to spend the day of your flight fighting statistics

In this case, it's my "this doesn't make any sense, I'll just look at the cdf and pull medians" vs "the stuff that doesn't make sense". My method looks more robust, which makes me wonder why the other way is even there.

This is a bad screenshot, but in my defense, I didn't know the egg was going to hatch somebody new.

They're renovating this gate, but I hope they keep the old chandelier.

Saturday: Sure, why not cram everything I had planned for the weekend in one day?

Earlier in the week, I looked at the museum's webpage, and it looked like a number of new exhibitions had opened since last time, so I decided that sounded like a thing to do today.

With lunch at the cafe.

They again did not have the pasta I wanted.

So I went with the filet mignon sandwich.

 It was pretty good, and they had decent tomatoes.  Still probably would have preferred the pasta, but this worked.

And I decided to get dessert too.

Most of the art is the same, but I knew they had one specific exhibition I was looking forward to:

Dragons and Tigers!

I love Asian tiger art.  I don't know exactly why, but it's one of my favorite genres.

Tiger!

Tiger!

Tiger!

Dragon!

Tiger!

Dragon and Tiger, in kimono print form!

Dragon fight!

Tiger fight!

Utagawa Kuniyoshi!

Dragon!

 I'm sad that it was only one room, but I get that this was kind of a niche exhibition.

And today was when I discovered that the museum has an Albers.

And then I remembered that my dad wanted me to take a picture of something at Punchbowl.  I couldn't, due to renovations, but I could still take pictures.

Punchbowl.

From the lookout facing south.

North.

East.

And then I drove to the middle of the island to eat dinner.

Somebody kept talking about Five Guys last week, with the convincing argument when told "I don't want burgers" that "then everybody can just get hot dogs and be happy with that."

Finally, I had a fully trained Magikarp that needed to die in the jump fights, so I took that chance to see how the jump height scales with JP.  The square root fit I used last time gave a terrible fit.  So, I went with the next good option: logarithms.

That fits the data better, although I'm pretty sure there has to be a low-JP taper.  Unfortunately, I can't probe that region without resetting the game, I don't think.  Maybe I'll install it on my phone so I can redo that regime.  This tournament data set meshed nicely with the set from last time, so I just used the union of the two.

• Maybe it's just because I read proper news, but I thought everyone knew that the Republicans were lying about having any health plan other than "poor people die."  How stupid do you have to be to actually believe their lies?
• This sums up pretty well why you'd have to be an idiot to mess with Rei.
• I needed a third link, and found this Etta Candy clip from the Wonder Woman movie release.
• Then I found this link about European elephant art after the fall of Rome on Twitter.

Wednesday: Today was garbage, and I hate everything.

First fun thing: my home computer suddenly became super sluggish, and the load shot up to 100.  I came home from work to see what had happened, and it had crashed completely, turning itself off.  From the logs, it appears that some asshole was trying to connect to ssh, and it was taking effort to reject all those connections.  That's at least my best guess, considering that everything else seemed fine after rebooting.

Although the log data of rejected connections doesn't point out an unusual amount of activity.  Although the sudden jump around July 13 corresponds to when I bought a domain name from Google.  Great.  Wonderful.

Second fun thing: guy who decides to a) stop right in front of a crosswalk, so I also stop, thinking he's stopping for a pedestrian, like you're supposed to, but no, this asshole is just bad at driving; and b) just turn left from the right lane, forcing me to stop so I don't kill us all.

Third fun thing: tumblr in general.  Why is it so hard for them to not break everything every other day?  The fact that the standard suggestion is "install xkit", because the site is so unusable that you need to use a third party extension to make it not a pain in the ass.  In other words, xkit is broken again, and I'm annoyed.

• Stephen Miller is a racist, but that's not a big surprise, given that he works for a corrupt administration that is full of racists.
• I completely missed this book, but it's cancelled now, so I'll have to pick up the trade.
• Very much this.
• I miss having a president who isn't a disgrace.

Tuesday: I only crashed a computer twice today.

Not like I was running stuff that I really wanted finished or anything.

I didn't think I'd have a picture for today, so I took this when I got home.  I totally wasn't trying to avoid the awkward thing of opening my door while my neighbor opens his door.

But, then I saw that this administration continues to be worse than useless, and is planning to investigate to see if white people are being discriminated against in college applications.  Which is dumb, so I did the standard thing of "asking the government about the data, since I'm pretty sure the government has the data" and:

Thanks, competent part of the government!

Lines show fall college enrollment by broad racial category, taken from this table.  Dots show total US demographics, taken from the last table of this wikipedia page, because I'd already parsed and formatted one government table, and this one already had the same broad categories.  The main caveat is that looking at the subsequent section, you can see that younger age categories are less white.  However, I was not able to find a full year/age/race census split, so it's hard to plot a "college age" population.  Still, broadly speaking, white people have historically been over-represented in fall college enrollment, as has the Asian/Pacific Islander category, although that over-representation drops when you note the recently available Two-or-more category.  Conclusion: this administration can't do an hour's worth of work to discover their thesis is likely incorrect.  Alternate conclusion:

I am disappointed that this image makes a return.

R bit (removing all the mistakes, and me learning about the wonder of reshape2.melt, which takes "wide" tables where each column is a category sampled at each index/year, and turns it into a "long" table with each value indexed by the index and category):

p306 = data.frame(read.csv("./tab306_T.P",header=TRUE,sep='\t'))

library(reshape2)

demo = data.frame(read.csv("./demo.dat",header=TRUE,sep='\t'))
demomelt = melt(demo,id.vars="year")

p306X = p306[, c("year", "White", "Black", "Hispanic", "AsianPacificIslander", "AmericanIndianAlaskaNative", "TwoOrMore")]
p306meltX = melt(p306X,id.vars="year")

ggplot()  + geom_line(data=p306meltX,aes(x=year,y=value,color=variable)) + geom_point(data=demomelt,aes(x=year,y=value,color=variable))
ggsave("college_demographics.png")

• Rabbit Rowlet.
• Dogs.
• Weather clustering.  I vaguely remember noting the "large gradient"/"low high temperature" correlation back when weatherspark was a thing.

And tumblr broke xkit again, so my usual evening flow of

1. Read RSS.
2. Save interesting items.
3. Read through saved items, unsaving them as I open in new tab.
4. Like reblog tumblr items as I close them.

doesn't work.

Monday: Ok, but I needed a figure.

I spent a large chunk of the day proofreading a paper (that is much less complete than I thought it would be), and figures 5-12 were all various correlation plots.  Are "correlogram" plots not common (in quotes because google doesn't converge on if that's the right name)?

The NSF data again, which is a terrible data set to use for this, but I needed something.

The idea being that you have N variables, and want to see how they map against each other, so you put them into a big grid so you can see how all the things match against all the other things.  I wrote a quick version years ago, and went with the R version, just because that was reasonably easy to do:

install.packages("corrgram")
library(corrgram)

corrgram(nsf,upper.panel=panel.pts,lower.panel=panel.ellipse,order=TRUE)

png("why.png")

I'm sure there's a snek version as well.  That reminds me that I need to add a "snek bits" tag.

• My initial guess would have been the kid had been a jerk to the squirrel.  I also would have said that the guy might have cut the video to just show the bite, and maybe he did something before that.  However, putting them both together leads me to the same conclusion as the health department; this isn't a normal squirrel, and it's probably sick.  The only time I've ever been bitten by a squirrel I gained the proportional strength and abilities of a squirrel it was a tiny nibble by a squirrel who was really excited about food, and missed when it tried to get some.
• Ok, if this had been any other site telling me to "not order a fancy burger," I'd ignore it and put it in the "don't tell me what to do" category.  The problem is, I watch their youtube channel, and just last month, they had a video that was "this is a super fancy burger, and you should totally order it if you can."
• Also, I'm pretty sure this qualifies as the end state of "don't tell me what to do" in relation to food.
• I like the raincoat one the best.
• I couldn't tell what was wrong when I watched the bootleg copy of the trailer, but it's obvious when someone says it.  Thanos looks weird without his hat.
• Nova and Quasar can both fly, and they're spending their time making horses angry?
• On the other hand, Patsy Walker, Hellcat, playing frisbee with the Hulk is exactly what I would expect.
• This has been all over my Twitter timeline.  I don't understand how hard this is for people.  "Well, we'll have to see what the writers come up with!"  Boom.  You've diffused the ship talk you don't want to deal with, and you don't make something that makes the shippers feel bad.
• In case it isn't clear, I'm talking about the fandom relationship shipping, not the logistics of transporting objects shipping.

Sunday: I don't know.

Mostly I didn't know what I wanted to eat.  I eventually decided to go to Kono's, but then as I was driving, I didn't want to stop, so I just kept going, deciding along the way to do the complement of yesterday's dinner.

I got the super big one, because my plan is to eat part of it for dinner.  My tummy is grumbly, so that's probably going to be right after I finish this post.

Then I stopped at the park there.

According to the calculator, the tide level was about two and a half feet higher than normal.

 That explains why this area that's never flooded was flooded.

Finally, I was reading a book, and it made the claim that if a trait is in 1/200 people, then knowing 200 people means you know somebody with that.  "That doesn't sound quite right," I thought to myself.  This is basically the same binomial distribution problem from earlier, just with the case of looking at N_obs = 0.  So, plotting this up for three different probabilities to see how P_zero changes with the number of trials:

Side note, I do not like how snek handles grids.  You can plot everything using the plot object, but you cannot set the grid spacing.  That's an axis parameter, and you have to create an axis object by creating a subplot in the plot, even if you only want to have one plot.  Why is there this level of obscuration?

Anyway, for the 200 case, knowing 200 people only means there's a ~37% chance of not knowing someone with the trait.  That's pretty large.  Then I thought, "this is just the same curve, scaled along the trials axis, isn't it?

Yep.  This only looks reasonable in the semilog scale.  It'd be nice to have more gridlines, but nope.

So the 50% probability is at like 69% of the (N_trials * Pexp = 1) value.  These are specific values, and there should be some way to say "this is exactly equal to something".  I spent way to long trying to sort it out in terms of a Gaussian approximation to the binomial distribution before realizing that it's actually a Poissonian approximation I should be using.  Whoops.  As N_trials becomes big, the mean goes to N*P, with variance N*P*(1-P).  As P goes to zero, (1-P) goes to unity, and the mean equals the variance, and both are equal to NP.  This becomes the Poissonian lambda, so at the NP = 1 point, the probability of observing zero is just 1 / e.  The 50% point is the inverse CDF, and I don't have an inverse incomplete upper gamma function, but estimating from the gamma function I do have puts the point around 0.69315.  Cool.  Answers that question no one cared about.

• If they succeed in their repeal, they will bankrupt some people and kill others.  The two sets are not exclusive.
• I chose the 1/50 above based on this article's estimate of the chance of catching a legendary after the battle, because this is literally the exact same binomial problem popping up today.  2%.  I'm not going to bother with it if the rate is that low.  The comments here sum up my thoughts pretty well.  This isn't worth it.
• A cat story.
• Hulk and Valkyrie.

Tuesday: My plan for today was to eat hot dogs, take a walk, eat more hot dogs, and watch some fireworks.

I did all those things, and did more editing for SciPy stuff.  I do not like how Snek handles Unicode, because that handling appears to consistently be "be as difficult about everything as possible."

I also drew a very bad picture of how different parts of a probability distribution are included in a calculation.

 And I battled more with my Magikarp, so I have more data to include in the plot from yesterday:

This is the "most useful" version.

This splits two data points into their true input JP and the 5% and 25% scaled values.  I had boosts on those two runs, so the scaled values are probably more correct.

I also need a better way to put code into blogs.  I've decided I'm going to copy my R code when I make things like this, so I can find them later.  It really is super easy:

j = data.frame(read.table("./jp",sep='\t',header=TRUE))
ggplot(j,aes(x=jp,y=height)) + geom_point() + stat_smooth(method='lm',formula=y ~ sqrt(x), col='red')
ggsave("/tmp/jp2.png")

And I took a walk:

The moon was out.

 I saw four geckos and one dog that threw up.  I was happier for one of those.

This didn't quite capture all the pinks and purples.

And then I came home and had my code insulted.  If the input data wasn't so bad, the code would do a better job on it.  More webpages should be designed with a structured data concept.

Then there were fireworks:

Boom!

Ka-ka-boom!

Ba-fszzzzzzzz.

Boom boom boom siziziziziz.

Boom ka boom!

Boo boom!

Boom boom boom boom!

Boom ka boom boom!

Boom boom boom boom boom boom boom boom!

Fazzzzzzzzzzzzzzzzzzz.

• Bee facts.
• Squirrel.
• Squirtle.
• I'm sad that it's ending, but he has to do what he thinks is best.
• I don't have anything to add.

Monday: I should have just taken the four day weekend. I would have been about as productive.

However, last night, I was able to plot up and fit how the height a  Magikarp jumps scales with JP:

JP/1000, obviously.  But my "this feels like it's a square root" idea seems to be supported.  This is just one pass through the league I'm currently battling.

And today I finally decided that I wasn't going to find an easy way to reshape this table of PhD recipients, so I just wrote a perl script to do it.  Once it was in nice "field/year/numbers" format, I was able to get R to make a pretty plot:

So I guess the 24% value probably isn't too far off.

f = data.frame(read.table("./matched.dat",sep='\t',header=TRUE))

ggplot(data=f,aes(x=year,y=(female / (female + male)), color=field)) + geom_line(aes(group=field)) + geom_point()

• A good story.
• "Millions of identical human men".
• Cocoa.
• I am glad to see that Squirrel Girl and Koi Boy being friends is backed up by nature.