I have seen a couple of approaches to this. Most involve setting the servo target to the servo value +/- a predefined step if a certain button is pressed. The general gotcha is that unless there is some sort of mutex and/or wait, the holding down a button for a fraction of a second can result in the target being set to a min/max value as the statement is executed rapidly inside the control loop. Remember that the servo's value is actually just the previously set target... not the actual/physical position (which it takes time to reach).
One design which just came to mind when I read your post was to create a new task whose sole responsibility is the control of the servo in question you wish to control this way. The task would check the buttons (I imagine you'd want two; one for each direction), if pressed check for min/max constraints, increment/decrement the servo, then sleep (via a wait statement) for a time long enough to allow the servo to physically move the distance the target was changed by.