Make a Live USB to boot from a USB drive.Some Linux distros are designed to fit on a small USB, but choose one that fits with the way you like to work. First, of course make sure your potential host machine will boot from USB
The proper way to remove USB devices in Windows is to use the Safe Removal option, but it can be a pain. If you have the quick removal policy set for your USB device, you can safely remove it without using Safe Removal.
If you want a portable solution of your Linux Machine, Linux Mini PC is the best solution for you. Because these are easy to handle,need low power consumption,fit on a small space and can operate with Open-Source Operating System. Not all Linux.
How to Install Multiple Bootable Operating Systems on One USB Stick