First, welcome! Welcome to this great community and hobby. I know I am not the only one that will tell you that this "hobby" will quickly turn into a passion and even obsession.
In regards to starting, I think there is a benefit to starting with extract just to get the process down, specifically the sanitation process. Although, I do think if you are in it for the long haul you can jump straight into all-grain. Know that there is a learning curve and some more equipment will be necessary unless you are doing BIAB.
If you have a local hombrew store, give them a visit. If not, lots of the big online stores (i.e. midwest, northern, etc.) sometimes have sales which might save you some on your first set up. Cheers and welcome!