Stochastic computing is a re-emerging computing paradigm, that expresses numbers as probabilities. By realizing them in the form of random bitstreams, it enables hardware-efficient calculations with high fault-tolerance. This paves the way for efficient, bit-flip-robust implementations, that can be operated on faulty hardware, in very noisy environments or with unreliable emerging technologies. The aim of this work is to provide a state-of-the-art overview of the computing paradigm and to apply it to a selected application, the Scaled Sparse Kaczmarz algorithm. We first present relevant concepts of stochastic computing as well as its building blocks (e.g. adders and multipliers). For this, we introduce a novel weight-based viewpoint, that simplifies the design and investigation of stochastic computing systems. Based on our investigations, we identified state-of-the-art adders to be a major limiting factor for practical implementations. They either inherently apply a scaling factor on their result or use conventional binary number representations, that make them vulnerable against bit flips. We solve these issues by introducing novel shift-register-based non-scaled adders, novel upscalers and a novel scalar product implementation. With its unique concepts, the scalar product implementation significantly outperforms state-of-the-art implementations, if accuracy requirements and input vector lengths are sufficiently high/large. In the second part of this work, we implement the Scaled Sparse Kaczmarz algorithm via stochastic computing. It is a scaled variant of the Fast Linearized Bregman iteration, a highly-efficient optimization algorithm, that is used for sparse estimation. On the way to the final implementation, we solve several challenging issues, such as introduced scaling factors and statistical dependence of bitstreams. We synthesize the design and present bit-true simulation results, that are compared to those of a double-precision floating-point reference. There is a good agreement of the results, demonstrating, that even powerful algorithms for sparse estimation can be realized as a stochastic computing system.