Today's ‪#‎PizzaADayDiet‬ occurred at Home Slice Pizza -- Don Tate joined me for the sausage, mushroom, and green pepper pie! This was the thickest thin crust I've had so far, and was sufficient to be not -floppy, yet not doughy, with a good, chewy texture. The cheese was flavorful and the toppings were each present in every bite.


Altogether, a most excellent pizza -- and they put the leftovers in a tinfoil swan (I've never seen that before in real life :-)).

Monday I'm going to be running in the 25th Capital of Texas Triathlon!  It's my first triathlon (Olympic distance) in twenty years and I'm pretty jazzed.

Steely-eyed determination 20 years ago. :-)

One of the great things about triathlons (and running races in general) is that you get to occupy unusual spaces: the last ones I did were Leon's Triathlon in Hammond, Indiana, a couple of Bud Light triathlons and others in Chicago. Leon's had a swim in Wolf Lake (shudder), followed by a cycle leg on an elevated highway that ran past the old U.S. Steel plant, and a run leg through an industrial downtown.  The Chicago ones were on the lakefront, just north of Navy Pier, with a bike on Lake Shore Drive and a run along the lake.

2013 CapTexTri

The CapTexTri also has a great location in downtown Austin, with a 1.5k swim in Lady Bird Lake; a 40k (24.8 mile) bike on a quadruple loop through downtown Austin, including Congress Avenue and Cesar Chavez; and a 10k (6.2 mile) run through Zilker Park.

The only thing I'm not too keen on is the bike route, since it requires you to do the same loop four times with a bunch of corresponding hairpin turns.  I don't like loop routes because I always think of how many more times I have to do the thing...Still, going up and down Congress Avenue without any cars is going to be pretty cool. As long as there are no poles in the middle of the road, I should be okay. :-).

Don't ask.

I feel fairly good about my training.  I've maintained good running mileage after the Austin Marathon and Austin Distance Festival and got some good workouts in even while traveling doing school visits.

On Stone Arch Bridge in Minneapolis

The swim is probably my weakest event -- I could stand to do more work on technique and probably do more open water swimming, but the distance won't be an issue.  Also, Lady Bird Lake isn't going to have waves (I seem to recall a couple of triathlons in Chicago with 3-4 foot waves on Lake Michigan (and this was on the near side of the breakwater).  Also, I won't have to deal with a wet suit.  My biggest concern is to not get kicked in the face. :-).

Lady Bird Lake during 2013 CapTexTri

The bike I'm feeling good about as well.  I'll be using the bike I used for my triathlons back in the day - a Trek 1000 I bought when I was in grad school for $450 (a guy at one bicycle shop here tried to sell me a new one, asking if I had a "nostalgic attachment" to it.). I do, but I also don't think a new bike is going to drastically transform my performance.  At least not $2000 worth :-). (A guy at another bike shop told me the Trek 1000 was his first road bike and he wished he still had it.  It's possible he was being kind :-)).

Tomorrow is packet pick-up, bike drop-off, and a chance to scope out the transition area, which I'll need because I can't see without my glasses...:-)

Oh, well.  Qapla!

So my plan to do my first triathlon in twenty years on Monday didn't turn out so well.

It rained.

A lot. But not so much in town.

At Camp Mabry, just north of central Austin, we got less than an inch of rain last Thursday. At Bergstrom Airport, just south and east of downtown, they got about nine inches.  And it was much worse farther east, along the Colorado River (which also runs through Austin).

On Friday and Saturday, it rained in the Hill Country.  West of Austin.  Upriver.

On Sunday, there was the CapTexTri expo and packet pickup and bike dropoff.

Swag!  My first cowboy hat since I was around seven.

Bike drop off.  I got there early.

Packet pickup

Coveting my neighbor's bike, Part I.

It was clear and sunny.  But because of all the rain, the LCRA opened at least one floodgate from the Tom Miller Dam, releasing water into Lady Bird Lake.  Which was where the swim portion of the CapTexTri was supposed to take place.

At the course talk at the expo, they announced that there was a flow of about 3 mph and they were considering changing the course so that it ran point to point (downriver).

Course talk.  Could've used a projector.

Later that day, they announced the swim was canceled.  Which was disappointing, but I've weathered a couple of triathlons in Chicago where that had happened and one when it probably should've.

Monday, race day, I awoke at 5 AM, fed the cats, ate breakfast and drank coffee, and then I heard rain.  Lots of rain.

Nevertheless (discovering, to my chagrin, that I am apparently an optimist), I headed out.  When I arrived at the transition area, I was told it was closed, and that we should shelter at Palmer Events Center or its garage.  This was around 6-620.

In the garage and on the deck of the Palmer Events Center, folks seemed to take things in stride and with humor.  Some people were concerned about hairpin turns on a wet course, but were generally willing to take it slow.

Sheltering in the garage

We heard a tentative plan to cut short the bike portion to 20k, but the rain and lightning continued. Finally, at around 830, race officials called off the bike portion entirely because of flooding on the course. 

Still a bit lightning-y

They announced that Olympic distance participants could do a 10k, while sprint participants could do a 5k and that start time would be at 10 am. Most folks removed their bikes and went home or back to their hotels. There was some grumbling -- last year the event had been cut short due to flooding, as well, and I gather there had been similar problems in 2014, too.

Athletes clearing out the transition area

I took my bike back to my car, but decided that I'd gotten up at five that morning to run a race and so, I was going to do one.  Besides, I didn't want to waste all those carbs I'd eaten in the past few days. :-).

At ten o'clock, the rain pretty much stopped.  And then we were off!  By 10:15, the sun came out.  No, really. 

Everyone who's still there seems in good humor :-)

And we're off!

Turned out, there were only about 200 of us who stuck around for the 10k, with another 150 for the 5k (out of around 3000 original participants), but everyone seemed to be having a good time.  I was pretty happy with my race -- I'm not sure it was exactly 10k, but I still did one of my better overall times and paces. At least this century :-).

The view from the Biergarten.  Note the utter absence of rain.

Sunny skies.

On the whole, it was a bit surreal but fun, although in the moment sometimes frustrating.  And, in retrospect, kind of funny.  I think the organizers did a good job under trying conditions and kept us pretty well informed via social media.  So, thanks (And I am really glad I wasn't in charge :-)).  Thanks also to all the volunteers who stuck around to the bitter end.

 Oh, and I actually ended up getting a bit of a tan.

Coveting my neighbor's bike, Part II

Epilogue: Late Monday, the Austin Fire Department closed Lady Bird Lake and Lake Austin to all boat traffic.

And they're giving us a discount for the 2017 race. :-).

Here's what the lake looked like Tuesday morning (Normally, there is no current at all):
















 

The gas giant is the Solar System's largest planet. Here's why it's also the fastest-spinning planet.

This was a big week in space, from Curiosity stumbling upon sulfur crystals to an exoplanet discovery and a major advocacy effort.

Jupiter has often been thought to protect the inner Solar System from asteroids and comets, but new research has shown that the giant planet may actually increase the risk of an impact.

NASA’s Europa Clipper spacecraft launched today aboard a SpaceX Falcon Heavy rocket from NASA’s Kennedy Space Center in Cape Canaveral, Florida.

Table of Contents

1. Thread Local storage exhausted
2. The layer of indirection
3. I can fix her
4. Let's write some tests!
5. Summary

Thread Local storage exhausted

In the last post I've described a technique to use dynamic variables by value instead of the name by utilizing the operator PROGV. Apparently it works fine on all Common Lisp implementations I've tried except from SBCL, where the number of thread local variables is by default limited to something below 4000. To add salt to the injury, these variables are not garbage collected.

Try the following code to crash into LDB:

(defun foo ()
  (loop for i from 0 below 4096 do
    (when (zerop (mod i 100))
      (print i))
    (progv (list (gensym)) (list 42)
      (values))))
(foo)

This renders our new technique not very practical given SBCL popularity. We need to either abandon the idea or come up with a workaround.

The layer of indirection

Luckily for us we've already introduced a layer of indirection. Operators to access dynamic variables are called DLET, DSET and DREF. This means, that it is enough to provide a kludge implementation for SBCL with minimal changes to the remaining code.

The old code works the same as previously except that instead of SYMBOL-VALUE we use the accessor DYNAMIC-VARIABLE-VALUE, and the old call to PROGV is now DYNAMIC-VARIABLE-PROGV. Moreover DYNAMIC-EFFECTIVE-SLOT used functions BOUNDP and MAKUNBOUND, so we replace these with DYNAMIC-VARIABLE-BOUND-P and DYNAMIC-VARIABLE-MAKUNBOUND. To abstract away things further we also introduce the constructor MAKE-DYNAMIC-VARIABLE

(defpackage "EU.TURTLEWARE.BLOG/DLET"
  (:local-nicknames ("MOP" #+closer-mop "C2MOP"
                           #+(and (not closer-mop) ecl) "MOP"
                           #+(and (not closer-mop) ccl) "CCL"
                           #+(and (not closer-mop) sbcl) "SB-MOP"))
  (:use "CL"))
(in-package "EU.TURTLEWARE.BLOG/DLET")

(eval-when (:compile-toplevel :execute :load-toplevel)
  (unless (member :bordeaux-threads *features*)
    (error "Please load BORDEAUX-THREADS."))
  (when (member :sbcl *features*)
    (unless (member :fake-progv-kludge *features*)
      (format t "~&;; Using FAKE-PROGV-KLUDGE for SBCL.~%")
      (push :fake-progv-kludge *features*))))

(defmacro dlet (bindings &body body)
  (flet ((pred (binding)
           (and (listp binding) (= 2 (length binding)))))
    (unless (every #'pred bindings)
      (error "DLET: bindings must be lists of two values.~%~
                Invalid bindings:~%~{ ~s~%~}" (remove-if #'pred bindings))))
  (loop for (var val) in bindings
        collect var into vars
        collect val into vals
        finally (return `(dynamic-variable-progv (list ,@vars) (list ,@vals)
                           ,@body))))

(defmacro dset (&rest pairs)
  `(setf ,@(loop for (var val) on pairs by #'cddr
                 collect `(dref ,var)
                 collect val)))

(defmacro dref (variable)
  `(dynamic-variable-value ,variable))

;;; ...

(defmethod mop:slot-boundp-using-class
    ((class standard-class)
     object
     (slotd dynamic-effective-slot))
  (dynamic-variable-bound-p (slot-dvar object slotd)))

(defmethod mop:slot-makunbound-using-class
    ((class standard-class)
     object
     (slotd dynamic-effective-slot))
  (dynamic-variable-makunbound (slot-dvar object slotd)))

With these in place we can change the portable implementation to conform.

#-fake-progv-kludge
(progn
  (defun make-dynamic-variable ()
    (gensym))

  (defun dynamic-variable-value (variable)
    (symbol-value variable))

  (defun (setf dynamic-variable-value) (value variable)
    (setf (symbol-value variable) value))

  (defun dynamic-variable-bound-p (variable)
    (boundp variable))

  (defun dynamic-variable-makunbound (variable)
    (makunbound variable))

  (defmacro dynamic-variable-progv (vars vals &body body)
    `(progv ,vars ,vals ,@body)))

I can fix her

The implementation for SBCL will mediate access to the dynamic variable value with a synchronized hash table with weak keys. The current process is the key of the hash table and the list of bindings is the value of the hash table. For compatibility between implementations the top level value of the symbol will be shared.

The variable +FAKE-UNBOUND+ is the marker that signifies, that the variable has no value. When the list of bindings is EQ to +CELL-UNBOUND+, then it means that we should use the global value. We add new bindings by pushing to it.

#+fake-progv-kludge
(progn
  (defvar +fake-unbound+ 'unbound)
  (defvar +cell-unbound+ '(no-binding))

  (defclass dynamic-variable ()
    ((tls-table
      :initform (make-hash-table :synchronized t :weakness :key)
      :reader dynamic-variable-tls-table)
     (top-value
      :initform +fake-unbound+
      :accessor dynamic-variable-top-value)))

  (defun make-dynamic-variable ()
    (make-instance 'dynamic-variable))

  (defun dynamic-variable-bindings (dvar)
    (let ((process (bt:current-thread))
          (tls-table (dynamic-variable-tls-table dvar)))
      (gethash process tls-table +cell-unbound+)))

  (defun (setf dynamic-variable-bindings) (value dvar)
    (let ((process (bt:current-thread))
          (tls-table (dynamic-variable-tls-table dvar)))
      (setf (gethash process tls-table +cell-unbound+) value))))

We define two readers for the variable value - one that simply reads the value, and the other that signals an error if the variable is unbound. Writer for its value either replaces the current binding, or if the value cell is unbound, then we modify the top-level symbol value. We use the value +FAKE-UNBOUND+ to check whether the variable is bound and to make it unbound.

#+fake-progv-kludge
(progn
  (defun %dynamic-variable-value (dvar)
    (let ((tls-binds (dynamic-variable-bindings dvar)))
      (if (eq tls-binds +cell-unbound+)
          (dynamic-variable-top-value dvar)
          (car tls-binds))))

  (defun dynamic-variable-value (dvar)
    (let ((tls-value (%dynamic-variable-value dvar)))
      (when (eq tls-value +fake-unbound+)
        (error 'unbound-variable :name "(unnamed)"))
      tls-value))

  (defun (setf dynamic-variable-value) (value dvar)
    (let ((tls-binds (dynamic-variable-bindings dvar)))
      (if (eq tls-binds +cell-unbound+)
          (setf (dynamic-variable-top-value dvar) value)
          (setf (car tls-binds) value))))

  (defun dynamic-variable-bound-p (dvar)
    (not (eq +fake-unbound+ (%dynamic-variable-value dvar))))

  (defun dynamic-variable-makunbound (dvar)
    (setf (dynamic-variable-value dvar) +fake-unbound+)))

Finally we define the operator to dynamically bind variables that behaves similar to PROGV. Note that we PUSH and POP from the thread-local hash table DYNAMIC-VARIABLE-BINDINGS, so no synchronization is necessary.

#+fake-progv-kludge
(defmacro dynamic-variable-progv (vars vals &body body)
  (let ((svars (gensym))
        (svals (gensym))
        (var (gensym))
        (val (gensym)))
    `(let ((,svars ,vars))
       (loop for ,svals = ,vals then (rest ,svals)
             for ,var in ,svars
             for ,val = (if ,svals (car ,svals) +fake-unbound+)
             do (push ,val (dynamic-variable-bindings ,var)))
       (unwind-protect (progn ,@body)
         (loop for ,var in ,svars
               do (pop (dynamic-variable-bindings ,var)))))))

Let's write some tests!

But of course, we are going to also write a test framework. It's short, I promise. As a bonus point the API is compatibile with fiveam, so it is possible to drop tests as is in the appropriate test suite.

(defvar *all-tests* '())

(defun run-tests ()
  (dolist (test (reverse *all-tests*))
    (format *debug-io* "Test ~a... " test)
    (handler-case (funcall test)
      (serious-condition (c)
        (format *debug-io* "Failed: ~a~%" c))
      (:no-error (&rest args)
        (declare (ignore args))
        (format *debug-io* "Passed.~%")))))

(defmacro test (name &body body)
  `(progn
     (pushnew ',name *all-tests*)
     (defun ,name () ,@body)))

(defmacro is (form)
  `(assert ,form))

(defmacro pass ())

(defmacro signals (condition form)
  `(is (block nil
         (handler-case ,form
           (,condition () (return t)))
         nil)))

(defmacro finishes (form)
  `(is (handler-case ,form
         (serious-condition (c)
           (declare (ignore c))
           nil)
         (:no-error (&rest args)
           (declare (ignore args))
           t))))

Now let's get to tests. First we'll test our metaclass:

(defclass dynamic-let.test-class ()
  ((slot1 :initarg :slot1 :dynamic nil :accessor slot1)
   (slot2 :initarg :slot2 :dynamic t   :accessor slot2)
   (slot3 :initarg :slot3              :accessor slot3))
  (:metaclass class-with-dynamic-slots))

(defparameter *dynamic-let.test-instance-1*
  (make-instance 'dynamic-let.test-class
                 :slot1 :a :slot2 :b :slot3 :c))

(defparameter *dynamic-let.test-instance-2*
  (make-instance 'dynamic-let.test-class
                 :slot1 :x :slot2 :y :slot3 :z))

(test dynamic-let.1
  (let ((o1 *dynamic-let.test-instance-1*)
        (o2 *dynamic-let.test-instance-2*))
    (with-slots (slot1 slot2 slot3) o1
      (is (eq :a slot1))
      (is (eq :b slot2))
      (is (eq :c slot3)))
    (with-slots (slot1 slot2 slot3) o2
      (is (eq :x slot1))
      (is (eq :y slot2))
      (is (eq :z slot3)))))

(test dynamic-let.2
  (let ((o1 *dynamic-let.test-instance-1*)
        (o2 *dynamic-let.test-instance-2*))
    (signals error (slot-dlet (((o1 'slot1) 1)) nil))
    (slot-dlet (((o1 'slot2) :k))
      (is (eq :k (slot-value o1 'slot2)))
      (is (eq :y (slot-value o2 'slot2))))))

(test dynamic-let.3
  (let ((o1 *dynamic-let.test-instance-1*)
        (exit nil)
        (fail nil))
    (flet ((make-runner (values)
             (lambda ()
               (slot-dlet (((o1 'slot2) :start))
                 (let ((value (slot2 o1)))
                   (unless (eq value :start)
                     (setf fail value)))
                 (loop until (eq exit t) do
                   (setf (slot2 o1) (elt values (random (length values))))
                   (let ((value (slot2 o1)))
                     (unless (member value values)
                       (setf fail value)
                       (setf exit t))))))))
      (let ((r1 (bt:make-thread (make-runner '(:k1 :k2))))
            (r2 (bt:make-thread (make-runner '(:k3 :k4))))
            (r3 (bt:make-thread (make-runner '(:k5 :k6)))))
        (sleep .1)
        (setf exit t)
        (map nil #'bt:join-thread (list r1 r2 r3))
        (is (eq (slot2 o1) :b))
        (is (null fail))))))

Then let's test the dynamic variable itself:

(test dynamic-let.4
  "Test basic dvar operators."
  (let ((dvar (make-dynamic-variable)))
    (is (eql 42 (dset dvar 42)))
    (is (eql 42 (dref dvar)))
    (ignore-errors
     (dlet ((dvar :x))
       (is (eql :x (dref dvar)))
       (error "foo")))
    (is (eql 42 (dref dvar)))))

(test dynamic-let.5
  "Test bound-p operator."
  (let ((dvar (make-dynamic-variable)))
    (is (not (dynamic-variable-bound-p dvar)))
    (dset dvar 15)
    (is (dynamic-variable-bound-p dvar))
    (dynamic-variable-makunbound dvar)
    (is (not (dynamic-variable-bound-p dvar)))))

(test dynamic-let.6
  "Test makunbound operator."
  (let ((dvar (make-dynamic-variable)))
    (dset dvar t)
    (is (dynamic-variable-bound-p dvar))
    (finishes (dynamic-variable-makunbound dvar))
    (is (not (dynamic-variable-bound-p dvar)))))

(test dynamic-let.7
  "Test locally bound-p operator."
  (let ((dvar (make-dynamic-variable)))
    (is (not (dynamic-variable-bound-p dvar)))
    (dlet ((dvar 15))
      (is (dynamic-variable-bound-p dvar)))
    (is (not (dynamic-variable-bound-p dvar)))))

(test dynamic-let.8
  "Test locally unbound-p operator."
  (let ((dvar (make-dynamic-variable)))
    (dset dvar t)
    (is (dynamic-variable-bound-p dvar))
    (dlet ((dvar nil))
      (is (dynamic-variable-bound-p dvar))
      (finishes (dynamic-variable-makunbound dvar))
      (is (not (dynamic-variable-bound-p dvar))))
    (is (dynamic-variable-bound-p dvar))))

(test dynamic-let.9
  "Stress test the implementation (see :FAKE-PROGV-KLUDGE)."
  (finishes                              ; at the same time
    (let ((dvars (loop repeat 4096 collect (make-dynamic-variable))))
      ;; ensure tls variable
      (loop for v in dvars do
        (dlet ((v 1))))
      (loop for i from 0 below 4096
            for r = (random 4096)
            for v1 in dvars
            for v2 = (elt dvars r) do
              (when (zerop (mod i 64))
                (pass))
              (dlet ((v1 42)
                     (v2 43))
                (values))))))

(test dynamic-let.0
  "Stress test the implementation (see :FAKE-PROGV-KLUDGE)."
  (finishes                             ; can be gc-ed
    (loop for i from 0 below 4096 do
      (when (zerop (mod i 64))
        (pass))
      (dlet (((make-dynamic-variable) 42))
        (values)))))

All that is left is to test both dynamic variable implementations:

BLOG/DLET> (lisp-implementation-type)
"ECL"
BLOG/DLET> (run-tests)
Test DYNAMIC-LET.1... Passed.
Test DYNAMIC-LET.2... Passed.
Test DYNAMIC-LET.3... Passed.
Test DYNAMIC-LET.4... Passed.
Test DYNAMIC-LET.5... Passed.
Test DYNAMIC-LET.6... Passed.
Test DYNAMIC-LET.7... Passed.
Test DYNAMIC-LET.8... Passed.
Test DYNAMIC-LET.9... Passed.
Test DYNAMIC-LET.0... Passed.
NIL

And with the kludge:

BLOG/DLET> (lisp-implementation-type)
"SBCL"
BLOG/DLET> (run-tests)
Test DYNAMIC-LET.1... Passed.
Test DYNAMIC-LET.2... Passed.
Test DYNAMIC-LET.3... Passed.
Test DYNAMIC-LET.4... Passed.
Test DYNAMIC-LET.5... Passed.
Test DYNAMIC-LET.6... Passed.
Test DYNAMIC-LET.7... Passed.
Test DYNAMIC-LET.8... Passed.
Test DYNAMIC-LET.9... Passed.
Test DYNAMIC-LET.0... Passed.
NIL

Summary

In this post we've made our implementation to work on SBCL even when there are more than a few thousand dynamic variables. We've also added a simple test suite that checks the basic behavior.

As it often happens, after achieving some goal we get greedy and achieve more. That's the case here as well. In the next (and the last) post in this series I'll explore the idea of adding truly thread-local variables without a shared global value. This will be useful for lazily creating context on threads that are outside of our control. We'll also generalize the implementation so it is possible to subclass and implement ones own flavor of a dynamic variable.

Olympics 2024



View Comic!

Full Article