Growing any fruit or vegetable requires some patience and love of gardening, but growing asparagus is really a thankless task for a few different reasons.