Georeference and warp a drone image based on sensor orientation and location using transform.from_gcps()


I have been flying some drone surveys over the ocean and need to properly project and georeference and warp my images. I have all the information I think I need: lat, lon, altitude, yaw, pitch, and roll along with sensor optics params. I have to imagine this can be done with some existing package and I had thought some rasterio functions were suitable, but I can't find any. So I've been using the cameratransform package to get the GPS positions of the corners of my image to feed into rasterio:
import cameratransform as ct
# camera parameters
cam = ct.Camera(ct.RectilinearProjection(focallength_mm=f,
# gps pts are lat lon
cam.setGPSpos(img.latitude, img.longitude, img.altitude)
# these are the coordinates of the image corners
coords = np.array([cam.gpsFromImage([0 , 0]), \
    cam.gpsFromImage([image_size[0]-1 , 0]), \
    cam.gpsFromImage([image_size[0]-1, image_size[1]-1]), \
    cam.gpsFromImage([0 , image_size[1]-1])])

From there I'm not certain what to do. I've tried basically saying these corners are GCPs and trying to warp the image in rasterio:
import rasterio
gcp1 = rasterio.control.GroundControlPoint(row=0, col=0, x=coords[0,1], y=coords[0,0], z=coords[0,2], id=None, info=None)
gcp2 = rasterio.control.GroundControlPoint(row=image_size[0]-1, col=0, x=coords[1,1], y=coords[1,0], z=coords[1,2], id=None, info=None)
gcp3 = rasterio.control.GroundControlPoint(row=image_size[0]-1, col=image_size[1]-1, x=coords[2,1], y=coords[2,0], z=coords[2,2], id=None, info=None)
gcp4 = rasterio.control.GroundControlPoint(row=0, col=image_size[1]-1, x=coords[3,1], y=coords[3,0], z=coords[3,2], id=None, info=None)
# Register GDAL format drivers and configuration options with a
# context manager.
with rasterio.Env():
    # open the original image to get some of the basic metadata
    with, 'r') as src:
        profile = src.profile
    # create rasterio transform
    tsfm = rasterio.transform.from_gcps([gcp1,gcp2,gcp3,gcp4])
    # I also tried this function but to no avail
    #tsfm = rasterio.warp.calculate_default_transform({"init": "epsg:4326"}),{"init": "epsg:4326"}), img.size()[0], img.size()[1], gcps=[gcp1,gcp2,gcp3,gcp4])
    crs ={"init": "epsg:4326"})
        transform = tsfm,
    with'example.tif', 'w', **profile) as dst:
        dst.write(, 1)

This does produce an image but it is not properly warped. It is just in the correct location. Since my sensor is a 40 deg off nadir the warping should be reasonably significant, when I warp it with cameratransform's cam.getTopViewOfImage() function I get this:

example image

So I'm curious if I'm missing some rasterio function or ability, isn't what the GCPs should be doing in the transform.from_gcps() function?