In a large stockpot, melt the butter over medium heat. Add the diced onion and celery, cooking for about 10 minutes, stirring occasionally until softened.
Sprinkle in the flour and whisk continuously for 2 minutes until it forms a roux. Gradually pour in the chicken broth while whisking to prevent lumps, ensuring a smooth mixture.
Stir in the sliced carrots, frozen peas, cubed potatoes, cooked chicken, salt, thyme, basil, and black pepper. Cover the pot and let it cook on medium heat for 20-25 minutes, stirring occasionally, until the vegetables are tender.
Once the vegetables are tender, lower the heat to a gentle simmer and keep the pot covered while you prepare the dumplings.
In a medium mixing bowl, whisk together the flour, dried dill, baking powder, and salt. Add the cold butter chunks and use a pastry blender (or your fingertips) to work it into the dry ingredients until the mixture resembles coarse crumbs.
Pour in the milk and stir gently until just combined—do not overmix.
Uncover the simmering pot and use a tablespoon to scoop portions of the dumpling dough, gently dropping them into the stew. Let the stew continue to simmer uncovered for 10 minutes, stirring occasionally to prevent sticking, until the dumplings are cooked through and tender.
Ladle the stew into bowls, making sure to include plenty of dumplings. Serve immediately and enjoy!