Library path issues: cannot find -l<library_name>

edit

I am getting the following error when running colcon build for a package in my ROS2-galactic workspace that references an external C++ library:

jdflo@jdflo-desktop:~/astro_ws$ colcon build
Starting >>> serialization
--- stderr: serialization                             
/usr/bin/ld: cannot find -lgtsam_unstable
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/graph_talker.dir/build.make:156: graph_talker] Error 1
make[1]: *** [CMakeFiles/Makefile2:139: CMakeFiles/graph_talker.dir/all] Error 2
make: *** [Makefile:146: all] Error 2
---
Failed   <<< serialization [0.27s, exited with code 2]

Summary: 0 packages finished [0.52s]
  1 package failed: serialization
  1 package had stderr output: serialization

I have set up my paths as follows (output from terminal):

jdflo@jdflo-desktop:~/astro_ws$ printenv | grep PATH
WINDOWPATH=2
AMENT_PREFIX_PATH=/home/jdflo/astro_ws/install/serialization:/opt/ros/galactic
CMAKE_PREFIX_PATH=/home/jdflo/astro_ws/install/serialization
COLCON_PREFIX_PATH=/home/jdflo/astro_ws/install
PYTHONPATH=/opt/ros/galactic/lib/python3.8/site-packages
LD_LIBRARY_PATH=/home/jdflo/lib/:/home/jdflo/lib/:/opt/ros/galactic/opt/yaml_cpp_vendor/lib:/opt/ros/galactic/opt/rviz_ogre_vendor/lib:/opt/ros/galactic/lib/x86_64-linux-gnu:/opt/ros/galactic/lib
PATH=/opt/ros/galactic/bin:/home/jdflo/miniconda3/condabin:/home/jdflo/.local/bin:/home/jdflo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/jdflo/julia-1.8.5/bin:/home/jdflo/julia-1.8.5/bin

Within my /home/jdflo/lib/ directory I have the missing lib.so files:

jdflo@jdflo-desktop:~/lib$ ls
cmake   libart_skills.so  libgtsam.so    libgtsam.so.4.2.0     libgtsam_unstable.so.4      libmetis-gtsam.so
gtwrap  libCppUnitLite.a  libgtsam.so.4  libgtsam_unstable.so  libgtsam_unstable.so.4.2.0

But colcon appears to be looking everywhere else in the paths except here (I assume since it never seems to search /home/jdflo/lib/. Could somebody help me figure out how to get colcon to search here? I have pasted my terminal output from trying to search with the linker:

jdflo@jdflo-desktop:~$ ld -lgtsam_unstable --verbose
GNU ld (GNU Binutils for Ubuntu) 2.34
  Supported emulations:
   elf_x86_64
   elf32_x86_64
   elf_i386
   elf_iamcu
   elf_l1om
   elf_k1om
   i386pep
   i386pe
using internal linker script:
==================================================
/* Script for -z combreloc -z separate-code */
/* Copyright (C) 2014-2020 Free Software Foundation, Inc.
   Copying and distribution of this script, with or without modification,
   are permitted in any medium without royalty provided the copyright
   notice and this notice are preserved.  */
OUTPUT_FORMAT("elf64-x86-64", "elf64-x86-64",
          "elf64-x86-64")
OUTPUT_ARCH(i386:x86-64)
ENTRY(_start)
SEARCH_DIR("=/usr/local/lib/x86_64-linux-gnu"); SEARCH_DIR("=/lib/x86_64-linux-gnu"); SEARCH_DIR("=/usr/lib/x86_64-linux-gnu"); SEARCH_DIR("=/usr/lib/x86_64-linux-gnu64"); SEARCH_DIR("=/usr/local/lib64"); SEARCH_DIR("=/lib64"); SEARCH_DIR("=/usr/lib64"); SEARCH_DIR("=/usr/local/lib"); SEARCH_DIR("=/lib"); SEARCH_DIR("=/usr/lib"); SEARCH_DIR("=/usr/x86_64-linux-gnu/lib64"); SEARCH_DIR("=/usr/x86_64-linux-gnu/lib");
SECTIONS
{
  PROVIDE (__executable_start = SEGMENT_START("text-segment", 0x400000)); . = SEGMENT_START("text-segment", 0x400000) + SIZEOF_HEADERS;
  .interp         : { *(.interp) }
  .note.gnu.build-id  : { *(.note.gnu.build-id) }
  .hash           : { *(.hash) }
  .gnu.hash       : { *(.gnu.hash) }
  .dynsym         : { *(.dynsym) }
  .dynstr         : { *(.dynstr) }
  .gnu.version    : { *(.gnu.version) }
  .gnu.version_d  : { *(.gnu.version_d) }
  .gnu.version_r  : { *(.gnu.version_r) }
  .rela.dyn       :
    {
      *(.rela.init)
      *(.rela.text .rela.text.* .rela.gnu.linkonce.t.*)
      *(.rela.fini)
      *(.rela.rodata .rela.rodata.* .rela.gnu.linkonce.r.*)
      *(.rela.data .rela.data.* .rela ...