Because every member of the straw hats turned luffy down at first, literally every single one, they say they're needed where they are, they can't leave, that their dream will have to stay as a dream, or in choppers case he needs more time and experience. This is the pattern, luffy finds an outcast invites them to join the crew, they tell him no or call him stupid or both, he keeps asking, the plot kicks in, they fight together, share their dreams, and join the crew.
