Wednesday, July 19, 2017

Wednesday: I don't think it's working, bird.

It's just a feeling.

Tuesday, July 18, 2017

Tuesday: I guess they're painting at the doctor's office?

Of course, I didn't realize this while going in.  "I wonder what they're doing to the floors."  Good job, me.

Monday, July 17, 2017

Sunday, July 16, 2017

Sunday: I have not been productive this weekend.

I just realized I was going to take the trash out today, but I never did.  So I guess that's a tomorrow job.  I think part of the issue is that it's been hot and muggy all weekend, and that makes doing things not fun.

I did get sushi.  It was really empty.


Saturday, July 15, 2017

Saturday: Well,

"Why not go to the Spalding House?"  So I do.  "Huh, why is the parking lot so empty?"  Go inside.  "Sorry, all the art stuff is closed for an exhibition switch.  The less cool outside stuff is still open."  So it's good that it's free when you have a membership.

Outside art stuff.

A tiny lizard friend.  Like super tiny.

Outside stuff.
So now what?
Pizza time.
And then I came home, and instead of going for a walk like I had originally planned, I watched youtube stuff.  Then remembered I could watch tomorrow's episode of the AH Game of Thrones play through, even though I don't really care about GoT.  The only problem of not watching on youtube is that I can't watch at double speed.  So.



Thursday, July 13, 2017

Thursday: Just before I went home, I realized that the hygenist did more actual work than the dentist.

Like, why not just have her do everything?  The dentist was kind of a jerk, and the only real "work" he did was stab me with the scrape-y thing and floss.  "Did she do the floss?" So she sometimes does that too?

And this was pretty much how work went.  Minus the tigers.  I added the tigers for fun.

I was also surprised at how much the Pobs value changes with Pexpect, but looking at the plot, it makes sense.  You have Nobs, and that's fixed, so if Pexpect goes up, that pulls the distribution upward, and the Pobs plummets.

I need to send more postcards.


Wednesday, July 12, 2017

Wednesday: No, it's fun to discover a massive problem with a database just when you want to go home.

But I think I was able to fix it with a redo, and changing all the "INSERT"s to "INSERT IGNORE"s.

And I wanted to make a proper version of the garbage scribble at the top of this post.

I really like the way R plots things.
 It's reasonably simple:

b = data.frame(N = seq(0,177))
b$P = choose(177,b$N) * 0.24**b$N * (1 - 0.24)**(177 - b$N)

L = rbind(c(0,0), subset(b, b$N <= 36),  c(36,0))
U = rbind(c(44,0), subset(b, b$N > 43), c(177,0))

ggplot(b,aes(x = N, y = P, color="P_expect = 0.24")) + \
  geom_polygon(data=L,aes(x = N, y = P, fill="Pobs")) + \
  geom_polygon(data=U,aes(x = N, y = P, fill="Pover")) + \
  geom_line() + geom_point()  + \
  scale_colour_manual(values=c("black"),name="",
                      guide=guide_legend(override.aes=aes(fill=NA))) + \
  scale_fill_manual(values=c("red","blue"),name="")

ggsave("/tmp/2017b.png")

The only major issues are that doing the shading requires constructing a polygon object, and that needs to have endpoints set correctly (or it shades the wrong way).  Getting all the colors and legend set was also not super obvious, and that "override.aes" thing is just nonsense.

"But it's not snek."  So then it was a challenge to see how to do this in snek, too.
Snek is simpler, but their documentation is far worse.  Examples should start slow, not alphabetically with "animation."  Why would you do that?
#!/usr/bin/env python3                                                                                                       
import matplotlib.pyplot as plt
import scipy.special
import numpy as np

P_expect = 0.24
N_talks  = 177
N_obs    = 36
N_expect = 44

N = np.arange(0,N_talks,1)
P = scipy.special.binom(N_talks,N) * P_expect**N * (1 - P_expect)**(N_talks - N)

# Lower portion
Nl = np.arange(0,N_obs,1)
Pl = scipy.special.binom(N_talks,Nl) * P_expect**Nl * (1 - P_expect)**(N_talks - Nl)
Zl = Nl * 0.0

# Upper portion
Nu = np.arange(N_expect,N_talks,1)
Pu = scipy.special.binom(N_talks,Nu) * P_expect**Nu * (1 - P_expect)**(N_talks - Nu)
Zu = Nu * 0.0

plt.grid()
label_text = "P_expect = %.2f" % P_expect
plt.fill_between(Nl,y1=Zl,y2=Pl,color="red",label="P_obs")
plt.fill_between(Nu,y1=Zu,y2=Pu,color="blue",label="P_over")
plt.plot(N,P,color="black",label=label_text)
plt.scatter(N,P,s=5,color="black")
plt.legend()
plt.savefig("mpl.png")

Having an explicit fill_between() function saved a lot of time.  The legend() function was also helpful for making it just work.

I'm still behind on my RSS stuff.

  • "Oh no!  This new Spider-Man movie confuses the timeline!"  I complain about stupid stuff, but this is too far.  Comic book timelines have been insane forever.  I mean, look at Squirrel Girl.  Doreen was 14 when she defeated Doom with Tony, then she did stuff for a few years with the GLA, then she was kind in the Avengers, then she babysat Luke Cage/Jessica Jones' daughter, and now she's in college.  How old is she?  Why isn't she 39 if the comics follow regular time?  Doesn't matter, she's doing college now, enjoy your wonderful stories.  Comic book time is meaningless.
  • Wonder Woman.
  • Whoops.
  • Best Spider-Man.
  • Buffalo.

Tuesday, July 11, 2017

Tuesday: My back hurts.

And I didn't have a chance to read all my internet today.  I did decide that since the company I use for domain name crap seems broken, I should just buy a domain name and use a proper service.  Googling that problem had Google tell me that Google does that.  Fine.  Done and done.  It doesn't look like the DNS update has propagated, but Googling that suggest it can take up to 24 hours.  Then I'll be able to easily read my internet while waiting for lunch again.  Woo.

Today's picture is the binomial distribution stuff that I took a picture of with my Google powered phone, from a company that was owned by Google when I bought it, sent via Google chat, and then uploaded to Google photos so I could use it on my Google blog.
The point is that a lot of the time, I just want crap to work, and Team G is pretty good at doing that.


Monday, July 10, 2017

Monday: I don't have anything cool to put here today.




Sunday, July 9, 2017

Sunday: Ugh, fine.

I decided I wanted Five Guys today.
So, you know, Five Guys.
It was done faster than they've ever made anything.  I didn't even have time to sit down before they called my number.

Then I drove back home, and eventually decided that I should probably go for a walk.  I didn't feel like driving down to Magic Island, so I just went around the neighborhood.  That's when I decided that I really don't like the new Pokemon Go update, because gyms now all have six people in them, and it's a pain in the ass to fight them all.  Plus, since they have these stupid raids running like all the time, even friendly gyms can't have anyone added half the time.  Basically, unless you have a group of people to play with, that entire part of the game is impossible.

Then I walked past the Korean place, thought about getting kalbi there for dinner, didn't, got home, and decided I really did want kalbi.

Also pancakes, so I drove down 90% of the way to Magic Island anyway.
¯\_(ツ)_/¯



Saturday, July 8, 2017

Saturday: Spidered Men

As said by Kraven the Hunter.
Who is not in the new Spider-Man movie.  However, I kept thinking about him, because Vulture (who is in the new Spider-Man movie) has a jacket that reminded me of him, because Kraven has that fur ruff on his vest.

Obligatory ticket picture.  With ok soup.
It's a pretty good movie.  Spider-Man gets a new start that makes sense given the rest of the Marvel movie stuff, Tony Stark keeps being a jerk, and everything mostly works out fine in the end.  I hope they have Ned in every Marvel movie forever, because he is probably the best character.  Donald Glover's "I've got ice cream in here" quote is good, and I'm glad that Aunt May isn't completely clueless.  I'm also wondering how much money was in the truck they backed up to Gwyneth Paltrow's house for her five minutes of screen time.

And it opened with the Squirrel Girl song:


And here's Doreen fighting Kraven:


"Why are you talking so much about Squirrel Girl instead of Spider-man?" Because she's awesome, duh.

Also Spider-Man didn't have his Spidey Sense, which is kind of a big thing to be missing.



Friday, July 7, 2017

Friday: Thanks, jerk in the pickup truck swearing in the street outside.

I didn't have a picture for today, and going out to see you shouting randomly before nearly crashing your truck (twice) made me try to take a picture of the moon.
It worked about the same as always.


I finally have my RSS saved items back below 1000.  I should really sort out the ~600 items that have been there for more than a year.

Thursday, July 6, 2017

Thursday: No seriously, this week is all messed up for me now.

I spent part of the day thinking it was Monday, because we had a meeting, and those are usually on Monday.  But it's not, so tomorrow I have to try and sort out things that I want to finish this week.

Also, I learned that the answer to "I have all this crap in a data frame, and I want answers for each subset, but I don't want to have to do any work" is ddply.  Originally I did this with work data, but using the NSF PhD statistics works as well.

library(plyr)

nsf = data.frame(read.table("./matched.dat",sep='\t',header=TRUE))
nsf$R = nsf$female / (nsf$male + nsf$female)
q = ddply(nsf, .(broad_field), function(x) c(m = mean(x$R),s = sd(x$R), quantile(x$R, c(0.0, 0.5, 0.75, 0.90, 0.98, 1.0))))
                           broad_field         m           s
1                            Education 0.6780287 0.012619293
2                          Engineering 0.2169928 0.014537864
3                  Humanities and arts 0.5035801 0.007278568
4                        Life sciences 0.5384433 0.017871779
5    Mathematics and computer sciences 0.2479207 0.009173072
6                               Otherb 0.5059506 0.011422802
7 Physical sciences and earth sciences 0.3117802 0.018656152
8       Psychology and social sciences 0.5832883 0.011162981

        50%       75%       90%       98%      100%
1 0.6816616 0.6865097 0.6927421 0.6930276 0.6930990
2 0.2220837 0.2279097 0.2304950 0.2320948 0.2324947
3 0.5058046 0.5084105 0.5096879 0.5120727 0.5126689
4 0.5453048 0.5518069 0.5544373 0.5564318 0.5569305
5 0.2466649 0.2529897 0.2614564 0.2629909 0.2633745
6 0.5105890 0.5143730 0.5165125 0.5207457 0.5218040
7 0.3140528 0.3249957 0.3339903 0.3353106 0.3356407
8 0.5849175 0.5890918 0.5946965 0.5973446 0.5980066

Ok, so the quantile data isn't super useful with the NSF data, but still.  Means and sigmas for each factor, and for the work data, pulling the quantile stuff was useful.  Then, hubris took hold, and I wondered if I use this to do linear fits to each factor as well.  The answer is no, not with ddply, because that reads and writes a data frame, and lm outputs a model object.  So you have to use dlply to save those models in a list, and then make a data frame from the stuff you care about in the list:

f = dlply(nsf, .(broad_field), lm, formula = R ~ year)
coeffs = ldply(f,coef)

coeffs$x2010 = (coeffs[,2] + 2010 * coeffs[,3]) * 100
coeffs$m = coeffs[,3] * 100

                           broad_field (Intercept)          year    x2010           m
1                            Education   -5.318890  0.0029835418 67.80287  0.29835418 
2                          Engineering   -7.627153  0.0039025598 21.69928  0.39025598 
3                  Humanities and arts   -1.145504  0.0008204401 50.35801  0.08204401 
4                        Life sciences   -9.438112  0.0049634604 53.84433  0.49634604 
5    Mathematics and computer sciences    1.055520 -0.0004017907 24.79207 -0.04017907 
6                               Otherb   -3.713211  0.0020990852 50.59506  0.20990852 
7 Physical sciences and earth sciences  -10.098766  0.0051793762 31.17802  0.51793762 
8       Psychology and social sciences   -4.194659  0.0023770883 58.32883  0.23770883 

And yes, x2010 is the same as the mean above, and m (percent change per year) is approximately 1/5 of the sigma above.  Taking a quick average of just the math, engineering, and physical sciences data for 2017 gives something like 28%.  The lack of improvement in math is a problem, and engineering starts at a deficit.  The depressing thing is that even with the highest improvement rate for the physical sciences, it's still something like 40 years until parity.

Also a plot:

library(ggplot2)
ggplot(nsf,aes(x=year,y=R,color=broad_field)) + geom_point() + geom_line() + geom_smooth(method=lm)
ggsave("/tmp/with_fits.png")

Yes, I'm sure there's a line thickness parameter I could have changed.  Whatever.

Wednesday, July 5, 2017

Wednesday: Mid-week holidays shouldn't be allowed.

They should just become long weekends.  Like Thanksgiving does.

While in California, I picked up the new Hawkeye trade, which follows young cool Hawkeye instead of old grumpy Hawkeye.  While reading, I realized something.  Kate's costume makes no sense.

From the cover.  I'm just noticing the art has the string on the wrong side of her arm.
Ok, having one sleeve kind of makes sense, since shooting a bow is going to cause bruises and scrapes if you don't cover up the arm holding the bow.  But what's with the holes?  Her shoulder and hips get super hot, and need to be cooled?

Apparently I didn't notice this throughout the entire previous series that I read while in France.
My best guess is that they're kind of like pockets.  ¯\_(ツ)_/¯  It's a good series.



Tuesday, July 4, 2017

Tuesday: My plan for today was to eat hot dogs, take a walk, eat more hot dogs, and watch some fireworks.

I did all those things, and did more editing for SciPy stuff.  I do not like how Snek handles Unicode, because that handling appears to consistently be "be as difficult about everything as possible."

I also drew a very bad picture of how different parts of a probability distribution are included in a calculation.
 And I battled more with my Magikarp, so I have more data to include in the plot from yesterday:
This is the "most useful" version.

This splits two data points into their true input JP and the 5% and 25% scaled values.  I had boosts on those two runs, so the scaled values are probably more correct.
I also need a better way to put code into blogs.  I've decided I'm going to copy my R code when I make things like this, so I can find them later.  It really is super easy:

j = data.frame(read.table("./jp",sep='\t',header=TRUE))
ggplot(j,aes(x=jp,y=height)) + geom_point() + stat_smooth(method='lm',formula=y ~ sqrt(x), col='red')
ggsave("/tmp/jp2.png")

And I took a walk:
The moon was out.
 I saw four geckos and one dog that threw up.  I was happier for one of those.
This didn't quite capture all the pinks and purples.
And then I came home and had my code insulted.  If the input data wasn't so bad, the code would do a better job on it.  More webpages should be designed with a structured data concept.

Then there were fireworks:

Boom!

Ka-ka-boom!

Ba-fszzzzzzzz.

Boom boom boom siziziziziz.

Boom ka boom!

Boo boom!

Boom boom boom boom!

Boom ka boom boom!

Boom boom boom boom boom boom boom boom!

Fazzzzzzzzzzzzzzzzzzz.


Monday, July 3, 2017

Monday: I should have just taken the four day weekend. I would have been about as productive.

However, last night, I was able to plot up and fit how the height a  Magikarp jumps scales with JP:
JP/1000, obviously.  But my "this feels like it's a square root" idea seems to be supported.  This is just one pass through the league I'm currently battling.
And today I finally decided that I wasn't going to find an easy way to reshape this table of PhD recipients, so I just wrote a perl script to do it.  Once it was in nice "field/year/numbers" format, I was able to get R to make a pretty plot:
So I guess the 24% value probably isn't too far off.
f = data.frame(read.table("./matched.dat",sep='\t',header=TRUE))
ggplot(data=f,aes(x=year,y=(female / (female + male)), color=field)) + geom_line(aes(group=field)) + geom_point()


Sunday, July 2, 2017

Sunday: And then I went home and discovered that I got the wrong damn size of lightbulb.

But first, hot pot.  This is apparently way more popular here than KBBQ, so I wanted to try it out.  This place is nice, because they have personal pots.

I'm not in general super worried about raw beef, but having a boiling cauldron to sterilize everything is good too.
It was good, but it's not as good as KBBQ.  There are more veggies available at hot pot, so it's "healthier," and the meat is AYCE, but it's also single meat.  There were gyoza and shumai on skewers, but then they don't get crispy (obviously).  Also they smuggle scalding broth into your mouth to attack you.

Also it's kaiten style:

See?
That makes it convenient to get a bunch of different veggies and sides.  I'll probably be back, but it didn't trigger as many good feelings as KBBQ.  Now I want kalbi again.



Saturday, July 1, 2017

Saturday: So I've decided that this weekend is "try all new restaurants" weekend.

Partially because I started yesterday with the kalbi, and partially because the place I was going to go for lunch doesn't do the Korean fried chicken and waffle anymore, and what use is that place, really, if they're going to stop making the best thing ever?

In any case, back in Alameda, I had another delicious Monte Cristo, so I spent a chunk of the morning browsing yelp.  Eventually, I found what looked like a decent version.

It's more "just a bar" than my picture of "gastropub." 
"Shouldn't the name have given it away?"  Yes, fine, but when I google "gastropub," I get something closer to what I have in my head: lots of tables, decent lighting, wacky decorations.  This is a lot more bar.  But whatever, the important thing is the foo:
P+J fries.  They're basically BBQ chips, but in fry form.
It's a bit much.  They're extra salty, and when I got outside (where there was light to see) I discovered that I had piles of BBQ dust under my fingernails.  Ick.  I still ate them all, but it's a bit much.
The main attraction.
 It's a bit greasy, but works pretty well.  Cafe Jolie is better, but this is probably my option when I want one here.

Then to the park to learn how much I hate the new Pokemon update.
Because it takes forever to take out gyms now.  There's always six opponents, and you have to punch them all like three or four times to make them go away.
Sure, I didn't have to use all of my team, but who cares.  Also raids just get in the way.  I don't want to fight a level 90382 Quilava.  I have a garbage fire otter, I don't need another one.  Why is this mid-tier jerk doing raids anyway?

And the gym didn't even change color when I took it over.  Great.
Not that it mattered, because while I was wasting time punching the other gym that's now there, I was kicked out, because unless you plan on playing with a team, apparently Niantic doesn't care about you.  Wonderful.  I earned one coin for the time I was in the gym, after spending like ten minutes fighting it.

Then it started to rain and my phone died before I could finish the second gym.  So I gave up, plugged in my phone, drove to Target, and had my phone die again when I got there, because I guess the car charger is enough to keep it going, but not enough to actually charge it.  Plus, Target no longer sells the stupid long fluorescent tube bulbs that go above my kitchen sink, so that trip wasn't even productive.

I bought popsicles, because I was angry, and Target had popsicles, and I didn't have popsicles, and I wasn't about to sit there and let stupid Target lord that shit over me when they can't even be bothered to sell the stupid light bulb I need.

Then I went home and watched internet crap until I was hungry again.  I had a different place selected, and google suggested walking.  Since I only did one lap at the park before my phone died, I went with it, but then they were stupid busy, so I ended up going to a different place, so that's what I'm going to talk about.
 
Bar Koko, the place that I think I've mentioned before, with some kind of note like, "I can see it from my couch."  It's a pretty good candidate for places to walk to for dinner.
And it has a bar, but less of one than P+J.
They're also not super speedy, but whatever.  There were three other groups in there for the entire time I was there, so it doesn't seem like they're too busy.
The hummus was pretty good.
I went with the "all the food" option.
It was a lot of food.
Another good option.  This is definitely going into the rotation.