First, if you expected that website speed influences your rankings big time, we need to clarify that this isn't so. While the two are related, you won't find a direct correlation – i.e. low ranking sites might load fast and vice versa.
The results depend on which metric you use to measure website speed. For instance, if you take into account the time needed to load the first byte of the page, then it turns out this is a huge factor because sites that are fast to load the first byte of a page typically rank higher in Google.
If you take other metrics into account – i.e. the time necessary to load the main content or the time it takes to load the complete page (with images and ads), then things change because it seems these two factors are not of that importance to Google.
Overall, it's believed that website speed has less than 100% effect on rankings. Google themselves have stated that they take website speed into account but they didn't disclose details about exactly what matters.
Nevertheless, many sites report the increase in traffic (from search engines or otherwise) after they optimize their site for speed. This is a pretty good reason to do the same if your site is slow – you are not doing it for Google, you are doing it for your users, your traffic, and your conversions. Website speed is key to user experience, so if your site is slow, there is no reason to keep it that way. What's the use of getting traffic from search engines, when your visitors have to hang for 10+ seconds before they can see your content – they will have left much before the page has loaded.