This is a good question, and one that is not super clearly explained in the documentation for either package (IMO). Contributions to the Wiki are definitely welcome.
For both cases, the system model is a generic motion model describing integration of acceleration into changes in position (i.e. a double integrator system). This model does not necessarily have much to do with the actual system you may be using the package with (it's ignoring the system-specific physics such as inertia, friction, nonlinear kinematics, etc.), and if you had a system where these generic motion models didn't do a great job of predicting the actual motion of your robot (e.g. a system with very fast dynamics and control loops) you may be better off writing your own filter. That said for many situations both of these packages can be setup to work pretty well.
The paper describing the robot_localization package says "For our application, ๐ is a standard 3D kinematic model derived from Newtonian mechanics." Here, f refers to the nonlinear state transition model. To me, that doesn't mean much, and I haven't spent a ton of time studying the equations in the actual implementation, but the function that is actually called to form predictions for the EKF is Ekf::predict(const double referenceTime, const double delta). A fast glance looks like they are assuming constant accelerations in all 6 DOF, integrating twice, adding Gaussian noise, wrapping angles, and returning updated state and covariance estimates.
For the robot_pose_ekf package, things are a bit simpler because they are assuming you have a mobile robot navigating in a plane (as opposed to arbitrary constant acceleration in 6D). The NonLinearAnalyticConditionalGaussianOdo::ExpectedValueGet() method is what is called to update the state prediction. This looks like a standard integration of a unicycle model (also called a kinematic car) with additive Gaussian noise. Here is where that function gets set as the system model, and here is where it is actually called when the filter is being updated.
