The fire selector detent spring is probably missing from the swap......you need to replace it (rule of thumb, put back things they way they came apart)
Watch this
video, skip to the 36sec mark, this is for a real ar-15. Just like the real ar-15, the WE m4 uses a detent spring for the fire selector. Find it, or buy a new one and follow the install instructions from the 36sec mark in that video