Showing posts with label snek bits. Show all posts
Showing posts with label snek bits. Show all posts

Tuesday, December 10, 2019

Tuesday: It turns out jupyter is good for writing demos

Because I can seemlessly fold SQL, bash, and snek into one document to show what's going on, without having to do much work.

I also should stop staying up late playing video games.

Standing on Leonidas's head.

Looking at burning ships that I may or may not have anything to do with.

Killing the Minotaur.

Tricking a land bear into becoming an ocean bear, something they are bad at.

Seeing Phoibe.

Catching on fire at Pericles' party.

Catching on fire again at the same donking party.

"""""Meeting""""" Alkibiades.

Friday, October 4, 2019

Friday: And now it's cold.

At least one of the math things I thought through over break works the way I wanted it to.  These plots are the distributions of 100k samples drawn from Poisson and Gaussian distributions with mean = variance = 8.5 * 10**i
So mean of 8.5.

Mean of 85.

Mean of 85000.
The Poissonian distribution turns into a Gaussian, and a +2-sigma deviation comes up ~2% of the time as you'd expect.  If this distribution is the number of times a thing happens a day, there should be like six of those kind of days a year.



Monday, August 26, 2019

Monday: Being back to work means being distracted by random things.

1) Stardew:
Oh geeze, I don't know?  This seems awfully fast, and I'm not sure I'm old enough to have a video game kid.

This, this isn't helping any!  Yikes.  Way to ramp up the pressure, Leah!
2) Twitter, lists, and those being used by bots to target attacks.

I saw this tweet yesterday, which made me think about how easy it should be to automatically block everyone who has put you on a list, and I was able to slap together this:

#!/usr/bin/python3

import twitter as tw
import time

api = tw.Api(consumer_key=CONSUMER_KEY,
             consumer_secret=CONSUMER_SECRET,
             access_token_key=ACCESS_TOKEN,
             access_token_secret=ACCESS_TOKEN_SECRET)

mm = api.GetMemberships(screen_name=SN, count=200)

ff = []
npage, ppage, new = api.GetFollowersPaged(screen_name=SN, count=200, cursor=-1)
ff.extend(new)

while npage != 0:
    time.sleep(1)
    npage, ppage, new = api.GetFollowersPaged(screen_name=SN,
                                              count=200, cursor=npage)
    ff.extend(new)
    print(npage, ppage, len(ff), ff[-1])

print(ppage, npage, new)
for idx, r in enumerate(mm): # .sort(key=lambda r: r['id'])):
    print(idx, repr(r))
    resp = api.CreateBlock(user_id = r.user.id)

which has key variables removed (for "don't spread your access tokens and secrets on your blog" reasons), and I think doesn't work due to incomplete testing (because I didn't want to auto-block however many people accidentally;  most things up to the block bit work fine if need to be rate limited more), but it seems like someone else would have come up with this already.

3) I hate when libraries don't have what seems like a critical function.

No comment.

4) How common was this?

This floorplan (along with a lot of others on that tumblr) have "maid rooms" included.  That made me wonder how often that happened.  From here, it sounds like 20-60%, depending on where in the country.  This says closer to 10%.  Still, wow.  Also: racism.

5) Other stuff.




Wednesday, August 14, 2019

Wednesday: Only half over?

That doesn't sound right.  I'm pretty sure I've been at this conference for forever at this point.  Still interesting, still learning stuff, still using things I've read about to make cool figures:
I can't figure out the transform code to flatten the orbits down into a plane, but I'm probably over looking something.
BBQ for the Team Dinner.  I like BBQ.

The gym.

And then some Codenames, which we lost the first game of because my team overthought.

Tuesday, July 30, 2019

Tuesday: It feels like this conference is killing me.

I'm super tired, and it feels like it's been going on forever, but it's actually only Tuesday.
I don't think that's legal, Team Rocket Grunt.

There's a gym here.

Monday, July 15, 2019

Monday: Back to work.

And already 2/3 things I'd like to check off fail tests, so I have to figure that out tomorrow.  Also cull my notes from SciPy into some useful list of interesting points.
I did learn that I can run CUDA code on my computer, although I'm not sure what this timing ration (CPU / GPU) tells me.  Do lots of the same things, so they can be parallelized, I guess.

Tuesday, July 9, 2019

Tuesday: conference tutorials

This time on tests, how to design easier to use unit tests with what's essentially a fuzzer, and how fuzzers are basically just Monte Carlo devices, and so can be used to solve problems if you're pathological.



Tuesday, July 2, 2019

Tuesday: I finished the API changes I wanted, but they still don't work.

So tomorrow is testing.

I also got the multiprocessing stuff to work for the snek rewrite of C code.  Twelve threads only gives a factor of two speed up, but that still left most of the CPU idle.  Maybe for the future I'll parallelize other bits, but now I should make the multi-fit work.

Or, you know.
I should probably stop playing it.  Or at least switch to a New Game+.  Maybe when I get back.


Monday, June 24, 2019

Monday: Meetings, outlining, discovering my last merge had bugs that no one caught. You know, a fun-day.

Ugh.  At least the bugs are easy to fix and cosmetic at worst, although ugh.  Also double ugh.

The outline worked well, though, and I was able to get parts A and B to output to the same format, which I'm calling a win.

Then I tried to find tiny cans of pop, and ended up going to Target because no place else sells them, I guess.  Then home, dinner, and another episode of CardCaptor Sakura.  Which led to this:
A relationship graph for all important human characters on the show.
Blue is for friendship, green is for "like-like", and red is for "hates."  I'm really unhappy that season three reworked some of the relationship things (redirecting the Sakura-Yukito-Xiaolong triangle; not redirecting the Rika-Terada-san creepfest).

But it gave me an excuse to play with a directed graph library, so that's always fun for the future.  It's a bit unstable to initial ordering/edges, but that's fine.  Snippet:

#/usr/bin/env python3

import pygraphviz as pgv

g = pgv.AGraph(strict=False,directed=True, concentrate=True)

char =  ['sakura', 'toya', 'yukito', 'xiaolong',
         'tomoyo', 'sonomi',
         'nadeshiko', 'fujitaka',
         'meiling',
         'naoko', 'chiharu', 'takashi', 'rika', 'terada-san']
for n in char:
        g.add_node(n)

g.edge_attr['color'] = 'green'

g.add_edge('sakura', 'tomoyo', dir='both', color='blue:green')
g.add_edge('toya', 'sakura')
g.add_edge('toya', 'xiaolong', color='red')
# etc...
g.draw("/tmp/ccs.png", prog='dot')


Finally, more Bloodstained, where I discovered I need another secret item to upgrade my sword again, which is a pain.  I also broke the game and fell out of the map, and then was greeted by "Party Legs" when I got back into the map:



Thursday, June 20, 2019

Thursday: I spent all day doing a review.

It was long and kind of complication, but the good news is that I now know how to do multiprocessing in snek with shared memory areas.

There's a dog to match the cat I showed yesterday.

Wednesday, April 3, 2019

Wednesday: pdb.set_trace = lambda: None 

Apparently that's the way to force your code to skip all the remaining debug breakpoints  It'd be nice if "clear" worked, but whatever.  "continue 9999" would also be great.

Routine gym takeover.
 And despite it being late, I did the whole "make dinner" thing.
Garlic noodles.  I think I've sorted out the recipe, which is nice because it's just "a gob of butter, a bunch of garlic paste, one teaspoon of everything else, as much parmesan as you can stand."  The arugula really helps, as the peppery flavor cuts through the garlicy goodness.  I'm also happy Target sells pre-cooked frozen shrimp, so I could claim this was a proper full meal.

Tuesday, September 11, 2018

Tuesday: "I'm going to save myself two minutes by demonstrating how cops are all basically terrible."

He decided that he really needed to turn left into the gas station to get something inside the shop, so he turned on the lights in his unmarked SUV.  Good job, asshole.  Glad you didn't have to wait for your slurpie.

Because it was easier to just crank this out in snek.
And then I spent the evening putting together slides for tomorrow, most of which are just images with a title, because I'm lazy, and words are hard.

Also I bought a new domain name today because it's shorter and easier to remember for other people, and I can use it to dump things I might need to share.  Like the slides I made for tomorrow.


Saturday, September 1, 2018

Saturday: I think this is my first day not leaving the house in forever.

It was actually kind of nice.  Wake up, go back to sleep for a lazy morning, think about getting up and making breakfast, go back to sleep some more, have a realization about the actual identity of the breakfast I was planning on having, read the internet for a bit, get up and clean the bathroom, get clean, answer the equivalent of the phone, do a quick tour of my apartment via phone, work on porting a FITS utility program I wrote ages ago to snek, discover that snek often has too much "not api" documentation, but that the api actually isn't totally stupid (I'm looking at you, CFITSIO), make lunch, make dinner, watch youtube, watch a show I've heard about until it got too cringey to continue, do a bit of work stuff to kick my tests for another day, fix the stupid bug that probably broke the tests yesterday, and I probably need to fill the dishwasher before I go to bed.

The revelation this morning was that the scrambled eggs + fried potatoes + pan sausage + sriracha dish from Julie is just just the insides of a breakfast burrito.  My plan was to use the sausage and sriracha to cover up my dislike of scrambled eggs, since I've always considered them gross.

That all changed today.  I've finally learned how to make scrambled eggs, and what has been wrong all this time.  I also cheated, so I'll include the cheats in my recipe:

  1. Cut up about two-three inches of green onion greens into super fine shreds.
  2. Beat two eggs with the green onions, then add a few splashes of soy sauce.
  3. Heat up a the small nonstick pan on medium
  4. Beat the eggs some more.
  5. No, really, like beat the eggs a lot more than that.
  6. Add a big chunk of butter to the pan, and immediately drop it to #2.
  7. Once the butter is melted, set the timer for two minutes.
  8. Add the eggs to the pan, swish it around, and then use the spatula to push the eggs to the opposite side.
  9. Repeat until ~1:30 have passed, and the eggs have stopped having liquid bits.
  10. Plate.
The other stuff just got fried up in the big nonstick pan.
And then I made burritos with all that stuff and some cheese, and they were delicious.  Plus I have a big container of sausage and potatoes in my fridge for tomorrow.  And, I now have a use for the remaining ~8 eggs in my fridge.  It's like a whole new world of food.

  • This was the only blog post I could find referencing the recipe.  I wish I had cider and apple butter.  People at work suggest apple butter is totally a thing here, though, so I just need to wait a month or two.
  • My base recipe for the eggs.  I should probably get mirin at some point so I can make proper tamagoyaki.
  • Today's youtube has been adversely affected by this crappy concert and their stupid advertisements.  I don't care, and I doubly don't care, even when you overlay the ad for the entire length of the video.
  • Aggretsuko.

Thursday, August 30, 2018

Thursday: Sprint planning.

I'm not sure if I like it, because it's kind of like a performance review and a meeting rolled into one, and I don't really like either.  At least I largely accomplished most of my August tasks, even though the expectation was that they'd take longer than a month.  And I signed up to present a talk at a future Wednesday meeting, because I'm bad at saying no.

Also today I discovered that my new doctor is really far away, but I think if I had ignored the google maps directions and went with the highway, it wouldn't have been that bad.  I've also officially not lost as much weight as I had thought, which is a bit annoying.

And I could have gotten Qdoba for lunch, but that would have involved turning around and turning around again, so I stopped at the similar place instead.
I think my more direct comparison is "Chipotle is spicier, and has nicer cuts of lettuce, but isn't as good in overall flavor.  Also too much cilantro."

Finally, I spent an hour putting together the start of a snek program that will do all of the FITS interaction that I want and had before scattered in a half dozen programs.  The snek interface is nicer than C, but I know it has speed and memory issues.  Still nice to go from blah to working version in such a short time.

Similarly, using snek to spam my RSS onto tumblr means I'm having trouble finding links each day.


Wednesday, August 30, 2017

Wednesday: Ugh, that just convolves the two

I know I've made this work before, but I can't seem to sort out what I'm doing wrong this time.

Maybe when I did this before, one of the two distributions was sufficiently close to flat that it didn't matter?

Ugh.  I guess interpolating the CDFs and working out ratios is the way to go.


Wednesday, July 26, 2017

Wednesday: Everything needs more and better documentation.

Earlier today, Julie IMed me and said something to effect of, "Drop whatever boring shit you're currently doing, and watch this youtube video."  So I did, and it was reasonably interesting, and showed me that I should rewrite some of my stats programs to use his more efficient standard deviation calculation.

When I got home, I attempted to install the package.  Finally, on the third computer I tried, I was able to get it to install and run.

Too bad it comes up with the wrong answer.

To be fair, this is a particularly difficult case, but it chose to do the split in the wrong dimension.  There's two closely located nearly Gaussian populations.  According to the documentation, you can specify different distributions for each dimension, but I couldn't get that to work, receiving the super helpful error message "ValueError: sample only has 2 dimensions but should have 1 dimensions".  The video suggested that it is able to take data that's partially classified, and use that along with the unclassified data to do better fits, so maybe tomorrow I'll see if I can figure out how to do that.  If I just say "these three are class A, and these three are class B", hopefully it'll get the dimensions correct.


Sunday, July 23, 2017

This is an elegant solution?

A = plt.axes()
A.set_yticks(np.arange(0,1.01,0.1))

That's silly.

A = plt.axes()
A.set_yticks(np.linspace(0,1,11))

Oh yes, that's much more obvious of a solution.

Sunday: I don't know.

Mostly I didn't know what I wanted to eat.  I eventually decided to go to Kono's, but then as I was driving, I didn't want to stop, so I just kept going, deciding along the way to do the complement of yesterday's dinner.
I got the super big one, because my plan is to eat part of it for dinner.  My tummy is grumbly, so that's probably going to be right after I finish this post.
Then I stopped at the park there.

According to the calculator, the tide level was about two and a half feet higher than normal.
 That explains why this area that's never flooded was flooded.

Finally, I was reading a book, and it made the claim that if a trait is in 1/200 people, then knowing 200 people means you know somebody with that.  "That doesn't sound quite right," I thought to myself.  This is basically the same binomial distribution problem from earlier, just with the case of looking at N_obs = 0.  So, plotting this up for three different probabilities to see how P_zero changes with the number of trials:
Side note, I do not like how snek handles grids.  You can plot everything using the plot object, but you cannot set the grid spacing.  That's an axis parameter, and you have to create an axis object by creating a subplot in the plot, even if you only want to have one plot.  Why is there this level of obscuration?
Anyway, for the 200 case, knowing 200 people only means there's a ~37% chance of not knowing someone with the trait.  That's pretty large.  Then I thought, "this is just the same curve, scaled along the trials axis, isn't it?
Yep.  This only looks reasonable in the semilog scale.  It'd be nice to have more gridlines, but nope.
So the 50% probability is at like 69% of the (N_trials * Pexp = 1) value.  These are specific values, and there should be some way to say "this is exactly equal to something".  I spent way to long trying to sort it out in terms of a Gaussian approximation to the binomial distribution before realizing that it's actually a Poissonian approximation I should be using.  Whoops.  As N_trials becomes big, the mean goes to N*P, with variance N*P*(1-P).  As P goes to zero, (1-P) goes to unity, and the mean equals the variance, and both are equal to NP.  This becomes the Poissonian lambda, so at the NP = 1 point, the probability of observing zero is just 1 / e.  The 50% point is the inverse CDF, and I don't have an inverse incomplete upper gamma function, but estimating from the gamma function I do have puts the point around 0.69315.  Cool.  Answers that question no one cared about.


  • If they succeed in their repeal, they will bankrupt some people and kill others.  The two sets are not exclusive.
  • I chose the 1/50 above based on this article's estimate of the chance of catching a legendary after the battle, because this is literally the exact same binomial problem popping up today.  2%.  I'm not going to bother with it if the rate is that low.  The comments here sum up my thoughts pretty well.  This isn't worth it.
  • A cat story.
  • Hulk and Valkyrie.

Wednesday, July 12, 2017

Wednesday: No, it's fun to discover a massive problem with a database just when you want to go home.

But I think I was able to fix it with a redo, and changing all the "INSERT"s to "INSERT IGNORE"s.

And I wanted to make a proper version of the garbage scribble at the top of this post.

I really like the way R plots things.
 It's reasonably simple:

b = data.frame(N = seq(0,177))
b$P = choose(177,b$N) * 0.24**b$N * (1 - 0.24)**(177 - b$N)

L = rbind(c(0,0), subset(b, b$N <= 36),  c(36,0))
U = rbind(c(44,0), subset(b, b$N > 43), c(177,0))

ggplot(b,aes(x = N, y = P, color="P_expect = 0.24")) + \
  geom_polygon(data=L,aes(x = N, y = P, fill="Pobs")) + \
  geom_polygon(data=U,aes(x = N, y = P, fill="Pover")) + \
  geom_line() + geom_point()  + \
  scale_colour_manual(values=c("black"),name="",
                      guide=guide_legend(override.aes=aes(fill=NA))) + \
  scale_fill_manual(values=c("red","blue"),name="")

ggsave("/tmp/2017b.png")

The only major issues are that doing the shading requires constructing a polygon object, and that needs to have endpoints set correctly (or it shades the wrong way).  Getting all the colors and legend set was also not super obvious, and that "override.aes" thing is just nonsense.

"But it's not snek."  So then it was a challenge to see how to do this in snek, too.
Snek is simpler, but their documentation is far worse.  Examples should start slow, not alphabetically with "animation."  Why would you do that?
#!/usr/bin/env python3                                                                                                       
import matplotlib.pyplot as plt
import scipy.special
import numpy as np

P_expect = 0.24
N_talks  = 177
N_obs    = 36
N_expect = 44

N = np.arange(0,N_talks,1)
P = scipy.special.binom(N_talks,N) * P_expect**N * (1 - P_expect)**(N_talks - N)

# Lower portion
Nl = np.arange(0,N_obs,1)
Pl = scipy.special.binom(N_talks,Nl) * P_expect**Nl * (1 - P_expect)**(N_talks - Nl)
Zl = Nl * 0.0

# Upper portion
Nu = np.arange(N_expect,N_talks,1)
Pu = scipy.special.binom(N_talks,Nu) * P_expect**Nu * (1 - P_expect)**(N_talks - Nu)
Zu = Nu * 0.0

plt.grid()
label_text = "P_expect = %.2f" % P_expect
plt.fill_between(Nl,y1=Zl,y2=Pl,color="red",label="P_obs")
plt.fill_between(Nu,y1=Zu,y2=Pu,color="blue",label="P_over")
plt.plot(N,P,color="black",label=label_text)
plt.scatter(N,P,s=5,color="black")
plt.legend()
plt.savefig("mpl.png")

Having an explicit fill_between() function saved a lot of time.  The legend() function was also helpful for making it just work.

I'm still behind on my RSS stuff.

  • "Oh no!  This new Spider-Man movie confuses the timeline!"  I complain about stupid stuff, but this is too far.  Comic book timelines have been insane forever.  I mean, look at Squirrel Girl.  Doreen was 14 when she defeated Doom with Tony, then she did stuff for a few years with the GLA, then she was kind in the Avengers, then she babysat Luke Cage/Jessica Jones' daughter, and now she's in college.  How old is she?  Why isn't she 39 if the comics follow regular time?  Doesn't matter, she's doing college now, enjoy your wonderful stories.  Comic book time is meaningless.
  • Wonder Woman.
  • Whoops.
  • Best Spider-Man.
  • Buffalo.