Emacs load performance

It appears as if Lispy mode is seriously slowing down my load times by invoking (require 'org) at load time. We can confirm this by adding the following code to our init file:

;; Confirm that lispy is the culprit
(with-eval-after-load 'org
  (debug))

(with-eval-after-load 'lispy
  (message "Lispy loaded."))

I don't see a way around this, unless I just drop lispy entirely.