I agree, but before you leave, you have nothing to loose if you ask why he dropped you after four days. Having an explanation may give you closure so you don't look back and second guess your decision. The explanation may or may not be the truth, only you can judge.
