For removing a particular payment method in particular  category  or particular product type you need to crate a even observer

1st You Need to create a module

app/code/Arvind/Cod

ist we need to register our module

<?php

\Magento\Framework\Component\ComponentRegistrar::register(
\Magento\Framework\Component\ComponentRegistrar::MODULE,
'Arvind_Cod',
__DIR__
);

Second We Need to create a two folder in our module  app/code/Arvind/Cod/etc

app/code/Arvind/Cod/Observer

in our etc folder we need to create three xml file

1 di.xml

2 events.xml

3 module.xml

di.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:ObjectManager/etc/config.xsd">

</config>

events.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:Event/etc/events.xsd">

<event name="payment_method_is_active">
<observer name="arvind_cod_observer_cod" instance="Arvind\Cod\Observer\Cod" />
</event>

</config>

module.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="../../../../../lib/internal/Magento/Framework/Module/etc/module.xsd">
<module name="Arvind_Cod" setup_version="1.0.0">
<sequence>
<module name="Magento_Backend"/>
<module name="Magento_Sales"/>
<module name="Magento_Quote"/>
<module name="Magento_Checkout"/>
<module name="Magento_Cms"/>
</sequence>
</module>
</config>

In Our Observer Folder  we have 1 File

Cod.php

<?php
namespace Arvind\Cod\Observer;

class Cod implements \Magento\Framework\Event\ObserverInterface
{
public function __construct() {}

/**
*
* @param \Magento\Framework\Event\Observer $observer
* @return void
*/
public function execute(\Magento\Framework\Event\Observer $observer)
{
$result = $observer->getEvent()->getResult();
$method_instance = $observer->getEvent()->getMethodInstance();
$quote = $observer->getEvent()->getQuote();

if(null !== $quote){
/* Disable All payment gateway exclude Your payment Gateway*/
if($method_instance->getCode() =='cashondelivery'){
$items= $quote->getAllVisibleItems();
foreach ($items as $eachitem) {
//$product = $eachitem->getById('id');
$catid = $eachitem->getProduct()->getCategoryIds();

if (in_array("3", $catid)){
$result->setData('is_available', false);
break;
}

}

}
}

}
}

for Product Type  just Changed

if(null !== $quote){
/* Disable All payment gateway exclude Your payment Gateway*/
if($method_instance->getCode() =='cashondelivery'){
$items= $quote->getAllVisibleItems();
foreach ($items= as $eachitem) {
if($eachitem->getProductType()=='downloadable'):
$result->isAvailable = false;
break;
endif;

}

}
}