Digital intervention tools against problematic smartphone usage help users control their consumption on smartphones, for example, by setting a time limit on an app. However, today's social media apps offer a mix of quasiessential and addictive features in an app (e.g., Instagram has following feeds, recommended feeds, stories, and direct messaging features), which makes it hard to apply a uniform logic for all uses of an app without a nuanced understanding of feature-level usage behaviors. We study when and why people regret using different features of social media apps on smartphones. We examine regretful feature uses in four smartphone social media apps (Facebook, Instagram, YouTube, and KakaoTalk) by utilizing feature usage logs, ESM surveys on regretful use collected for a week, and retrospective interviews from 29 Android users. In determining whether a feature use is regretful, users considered different types of rewards they obtained from using a certain feature (i.e., social, informational, personal interests, and entertainment) as well as alternative rewards they could have gained had they not used the smartphone (e.g., productivity). Depending on the types of rewards and the way rewards are presented to users, probabilities to regret vary across features of the same app. We highlight three patterns of features with different characteristics that lead to regretful use. First, "following"-based features (e.g., Facebook's News Feed and Instagram's Following Posts and Stories) induce habitual checking and quickly deplete rewards from app use. Second, recommendation-based features situated close to actively used features (e.g., Instagram's Suggested Posts adjacent to Search) cause habitual feature tour and sidetracking from the original intention of app use. Third, recommendation-based features with bite-sized contents (e.g., Facebook's Watch Videos) induce using "just a bit more," making people fall into prolonged use. We discuss implications of our findings for how social media apps and intervention tools can be designed to reduce regretful use and how feature-level usage information can strengthen self-reflection and behavior changes.