High redshift star forming galaxies are complex multi-phase systems where hot interstellar gas, cold dense molecular clouds and energetic particles (i.e. cosmic rays) all coexist. These starburst galaxies are especially likely to be environments abundant in energetic cosmic rays due to the presence of massive stars and their remnants. Stellar remnants can supply seed particles and generate the shocks (via supernova explosions and other violent events) needed to accelerate the seeds to very high energies.  This talk considers the interplay and interactions between the ambient partially ionized gases, dense clouds, and the energetic non-photonic cosmic ray particles in these environments. I will present the energy deposition rates by cosmic rays as they propagate though their host galaxy and beyond, accounting for their high-energy hadronic interactions with interstellar matter and the influence of developing galactic magnetic fields. I will discuss the astrophysical implications on the host galaxy and its circum-galactic environment, and outline possible mechanisms by which cosmic ray feedback may operate.